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Abstract 

Educational  institutions  at  all  levels  are  increasingly  examining 

the  adveuitages  of  computer-based  instruction  (CBI)  to  augment  or  replace 

conventional  classroom  learning  environments.  This  research  measures 

the  effectiveness  and  efficiency  of  a  CBI  program  in  relation  to  the 

same  course  content  delivered  in  a  conventional  classroom  mode  of  an 

undergraduate  course  that  teaches  students  the  basic  concepts  and 

techniques  of  automated  (electronic)  spreadsheets.  A  CBI  program  was 

created  to  >‘mirror^  the  in-class  instructional  material  of  the  course. 

The  performance  of  the  students  who  too)<  the  course  by  (TBI  was  compared 

to  the  performance  of  the  students  who  took  the  course  in  the 

conventional  mode.  The  CBI  course  was  found  to  be  significantly  more 

efficient  while  producing  learning  effects  similar  to  the  conventional 

mode  of  instruction.  The  students'  prior  experience  and  knowledge 

levels  were  offset  by  learning  in  either  instructional  mode.  The 

c  C. 

initially  ‘'weaker"  students  were  not  identifiably  weaker  after 
completing  the  course  by  either  method.  Self-reported  prior  experience 
was  an  effective  indicator  of  the  students’  actual  pre-course  knowledge 
level,  but  not  an  indicator  of  the  students'  post-course  performance. 


AN  EXPERIMEJrTAL  OTJDY  IN  THE  USE  OF  COMPUTER-BASED  INSTRUCTION 
TO  TEACH  AUTOMATED  SPREADSHEET  FUNCTIONS 

I .  Introduction 

General  Issue 

Computer-based  instruction  (CBI)  effectiveness  and  efficiency  have 
been  studied  extensively.  Educational  institutions  at  all  levels  eire 
increasingly  examining  the  advantages  of  CBI  to  augment  or  replace 
conventional  classroom  learning  environments. 

One  of  these  institutions  is  the  Air  Force  Institute  of  Technology 
(AFIT) .  "With  technology  changing  almost  daily,  the  need  to  keep 
education  current  while  anticipating  future  needs  is  a  demanding  role 
for  the  Air  Force  Institute  of  Technology"  (7:2).  In  its  continuing 
commitment  to  education  and  technological  superiority  (7:iv),  AFIT  is 
investigating  the  potential  benefits  of  computer-based  instruction. 

One  potential  candidate  course  for  CBI  is  the  School  of  Systems 
and  Logistics'  course  QMGT  290,  Introduction  to  AFIT  Computer  Systems. 
CM5T  290  is  an  undergraduate  summer  short  term  course  that  introduces 
students  to  the  computer  systems  they  will  be  using  in  their  graduate 
program.  An  eight  hour  unit  of  QMGT  290  introduces  students  to  the 
basic  concepts  and  techniques  of  automated  spreadsheets.  Student 
performance  in  the  spreadsheet  unit  is  measured  by  a  final  spreadsheet 
assignment  that  is  cstomarily  graded  on  a  satisfactory-unsatisfactory 
basis.  The  spreadsheet  unit  of  QMGT  290  is  currently  taught  using 
Quattro  Pro™  spreadsheet  software  on  IBM™  compatible  computer  systems. 
AFIT  provides  students  with  sufficient  IBM  compatible  computer  systems 
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loaded  with  Quattro  Pro  software.  In  addition,  many  students  own  IBM 
ccMjpatibie  computers  for  home  use. 

Specific  Problem 

The  specific  problem  of  this  research  is  summarized  in  the 
following  research  question:  In  one  of  AFIT's  curriculi,  what  are  the 
effectiveness  and  efficiency  of  learning  from  a  computer-based 
instruction  program  in  relation  to  the  same  course  content  delivered  in 
a  conventional  classroom  mode? 

Research  Objectives 

The  objectives  of  this  research  were  to  develop  a  computer-based 
instruction  program  to  introduce  fundamental  automated  spreadsheet  func¬ 
tions  and  measure  the  program's  effectiveness  and  efficiency  in  relation 
to  the  same  course  content  delivered  in  the  conventiona;!  classroom  mcde 
of  the  School  of  Systems  and  Logistics  course  QMGT  290. 

Steps  Taken  to  Address  the  Research  Objectives 

To  address  the  research  objectives,  the  following  sub-objectives 
were  met: 

1.  Develop  a  theoretical  knowledge  base  on  CBI  effectiveness  and 
efficiency,  and  examine  the  literature  on  CBI  design,  programming,  and 

human-computer  interface  features. 

2.  Identify  the  course  requirements  of  the  spreadsheet  unit  of 

the  School  of  Systems  and  Logistics  course  CHTr  290. 

3.  Determine  an  appropriate  experimental  design  to  use  for  this 

study , 


2 


4.  Create  and  verify  the  content  and  operation  of  a  CBI  program 
that  mirrors  the  cotirse  requirements  of  the  spreadsheet  unit  of  QMGT 
290. 

5.  Conduct  the  CBI  experiment. 

6.  Analyze  the  results,  and  state  the  findings  of  the  experiment 
and  the  literature  review. 

Definition  of  Terms 

Computer-Based  I nst rue t i on .  Computer-based  instruction  is  an 
educational  tool  that  uses  computers,  computer  programming,  and  human 
computer  interface  techniques  to  aid  or  support  the  education  or 
training  of  people  (8:87) .  For  the  purpose  of  this  study,  the  term 
"computer-based  instruction”  encompasses  all  of  the  terms  shown  in  the 
second  and  third  column  of  Table  1. 

CBI  programs  can  be  placed  under  one  of  three  major  categories: 
automated  drill  and  practice  (4:34;  5:110;  15:104;  32:43),  interactive 
tutorial  programs  (5:110;  9:104;  15:104;  18:529).  and  simulations  (5; 
110:  15:104;  18:529). 

For  this  research,  computer-based  instruction  is  defined  as  a 
stand  alone  (no  htiman  instructor),  interactive  tutorial  program. 

Instructional  Method  Effectiveness.  In  this  report,  instructional 
method  effectiveness  is  defined  as  a  measurement  of  student  achievement 
or  understanding  as  demonstrated  ir  a  post- instruct ion  assignment . 

Instructional  Method  Efficiency.  Instructional  method  eff..  lency 
is  defined  here  as  the  amount  of  direct  instructional  time  required  to 
accomplish  the  course  objectives.  The  efficiency  of  the  traditional 
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TABLE  1 


TE3?MS  USED  TO  DESCRIBE  THE  COMPUTER’S  INTERACTION  WITH 
EDUCATION  OR  TRAINING 


canputer 


-aided 

-assisted 

-augmented 

-based 

-extended 

-managed 

-mediated 

-monitored 

-related 


training 

instruction 

learning 

teaching 

education 


(11:295) 

"in-class"  instructional  method  is  the  amount  of  time  the  students 
spend  in  class  to  accomplish  the  course  objectives.  The  efficiency  of 
the  computer-based  method  of  instruction  is  the  amount  of  time  the 
students  spend  using  the  computer-based  instruction  program  to 
accomplish  the  same  course  objectives  as  the  in-class  instruction. 

Spreadsheet  Functions.  For  this  research,  spreadsheet  functions 
are  concepts,  techniques,  or  operations  that  are  required  for  a  basic 
operational  knowledge  and  which  allow  a  more  effective  and  efficient  use 
of  the  capabilities  of  an  automated  spreadsheet  package. 

Scope 

The  findings  of  this  study  are  limited  to  AFIT  graduate  students 
taking  the  spreadsheet  unit  of  the  School  of  Systems  and  Logistics 
course  QVJT  290  and  are  not  intended  to  directly  predict  how  computer- 
based  instruction  would  function  with  other  types  of  course  material  or 
student  populations. 
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Organization  of  this  Research  Document 


Chapter  II  of  this  document  contains  the  methodology  used  to 
address  the  research  sub-objectives  previously  outlined.  Chapter  III 
describes  the  research  findings.  Chapter  IV  provides  conclusions  and 
recommendations  based  on  the  research  findings. 
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II.  Methodology 


Overview 

This  chapter  descrit)es  the  methodology  used  to  answer  the  research 
objectives  posed  in  Chapter  I.  Each  sub-objective's  methodology  is 
addressed  separately. 

Sub-obiective  1 

Develop  a  theoretical  knowledge  base  on  CBI  effectiveness  and 
efficiency,  and  examine  the  literature  on  CBI  course  design, 
programming,  and  human-computer  interface  features. 

CBI  Effectiveness  and  Efficiency.  A  literature  review  of  past  CBI 
studies. was  conducted  to  provide  the  researcher  a  theoretical  knowledge 
base  on  the  effectiveness  and  efficiency  of  CBI.  A  summary  of  the 
applicable  findings  from  the  literature  is  provided  in  Chapter  III. 

CBI  Course  Design.  Programming,  arvi  Human-Computer  Interface 
Features .  In  order  to  create  an  efficient  and  effective  CBI  program,  a 
knowledge  of  successful  CBI  design,  programming  and  human-computer 
interface  features  was  required.  The  features  used  in  the  creation  of 
the  CBI  program  for  this  research  were  identified  through  a  literature 
review  and  are  detailed  in  Chapter  III. 

Sub-object iye  2 

Identify  the  course  requirements  of  the  spreadsheet  unit  of  the 
School  of  Systems  and  Logistics  course  QMGT  290. 

This  research  sub-objective  was  necessary  to  identify  the  specific 
course  requirements  of  the  spreadsheet  unit  of  OIGT  290.  to  ensure  that 
the  CBI  program  course  material  was  a  "mirror  image"  of  the  course 
material  presented  to  the  in-class  students.  This  sub-objective  was 


6 


addressed  through  two  sources:  a  survey  of  the  QMGT  290  coijise  director 
and  instructors  and  course  requirements  identified  during  the  previous 
offering  of  QMGT  290. 

The  (^<GT  290  Course  Director  and  Instructor  Survey.  A  copy  of  the 
course  director  and  instructor  survey  is  provided  at  Appendix  A.  A 
personal  interview  survey  was  developed  to  solicit  the  specific  course 
requirements  of  the  spreadsheet  unit  of  QMGT  290.  The  survey's  contents 
were  derived  from  an  interview  session  with  two  experienced  QMGT  290 
instructors,  and  from  the  Quattro  Pro  User's  Guide  (25) .  The  survey 
identified  open-ended  theoretical  knowledge  elements  and  mechanical 
knowledge  elements  that  instructors  believe  are  important  to  the  course. 
Theoretical  knowledge  elements  addressed  concepts  and  techniques  that 
the  respondents  felt  were  important  to  any  spreadsheet  usage,  regardless 
of  the  internal  capabilities  of  the  particular  spreadsheet  package. 
Mechanical  knowledge  elements  solicited  techniques,  functions,  or 
operations  that  were  directly  related  to  the  internal  capabilities  of 
the  Quattro  Pro  spreadsheet  package.  The  theoretical  and  mechanical 
knowledge  element  responses  helped  determine  the  specific  course 
material  that  would  be  presented  to  the  in-class  students,  and  thus 
included  in  the  CBI  program. 

A  draft  of  the  survey  was  presented  to  the  (^IGT  290  course 
director  for  recommendations  and  approval.  EXiring  administration  of  the 
survey,  the  instructors  who  would  be  teaching  (^<GT  290  during  the 
research  period  and  the  course  director  were  asked  to  rate  each  survey 
element  that  they  considered  to  be  a  290  course  requirement  with  a 

relative  importance  rating  of  "1."  All  elements  receiving  a  unanimous 
rating  of  "1"  were  included  as  course  requirements.  Elements  not 
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receiving  a  unanimous  rating  of  "1"  were  personally  discussed  with  the 
instructors  and  course  director  to  determine  inclusion.  Chapter  III 
provides  a  list  of  the  finalized  course  requirements.  All  of  the 
finalized  course  requirements  were  incorporated  into  the  computer-based 
instruction  program. 

The  Previous  QMGT  290  Course  Offering.  The  course  requirements 
from  the  previous  offering  of  QMGT  290  were  determined  through  an 
interview  with  an  experienced  QMGT  290  instructor.  These  requirements 
corroborated  the  requirements  identified  by  the  instructor/course 
director  survey. 

Sub-objective  3 

Determine  an  appropriate  experimental  design  to  use  for  this 

study . 

The  Experimental  Population.  The  population  for  the  experiment 
was  the  students  who  were  taking  QMGT  290  during  the  research  period. 

The  Experimental  Design.  IXie  to  its  simplicity,  ease  of 
adaptation  to  the  current  QMGT  290  instructional  method,  and  the 
requirement  to  permit  voluntary  participation,  a  self-selected 
experimental  group  quasi experimental  design  was  used  (10:126-127). 
Students  who  would  take  CJIGT  290  during  the  research  period  were  allowed 
to  voluntarily  participate  in  the  experiment.  Students  volunteered  to 
take  the  course  by  either  the  traditional  in-class  method  of  instruction 
or  the  computer-based  method  of  instruction.  The  students  who 
volunteered  for  the  traditional  in-class  method  of  instruction  became 
the  "in-class"  group.  The  students  volunteering  for  the  computer-based 
method  of  instruction  became  the  "CBI"  group.  The  students  were  further 
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sub-grouped  based  on  their  responses  on  a  survey  questionnaire  and  their 
score  on  a  pre-course  spreadsheet  knowledge  test.  Specifics  on  these 
sub-groupings  are  discussed  later  under  Sub-objective  6.  The  relevant 
characteristics  of  the  student  groups  are  discussed  in  Chapter  III. 

This  quasiexperimental  design  allowed  for  measurement  of  the  "main 
effect"  as  well  as  the  "interaction  effect"  of  the  grouping  variables 
(10:123) . 

Identification  of  Student  Characteristics  that  Could  Affect  the 
Eb<perimental  Findings.  A  literature  review  of  previously  conducted  CBl 
experiments  and  studies  identified  student  characteristics  that  could 
potentially  confound  the  experimental  results.  To  identify  potentially 
confounding  characteristics  of  the  stixients  participating  in  this 
experiment,  a  closed-response,  self-administered  questionnaire  was 
developed  (10:218:  13:64).  A  draft  of  the  survey  questions  was 
distributed  to  a  panel  of  three  experienced  spreadsheet  users,  the  QMtTT 
290  course  director,  and  an  AFIT  faculty  member  not  directly  involved 
with  QMGT  290  for  face  validation.  The  survey  was  modified  based  on  the 
pane 1 ' s  recommendat i ons . 

Tfie  survey  (referred  to  as  the  pre-QHGT  290  student  survey  for  the 
remainder  of  this  report)  was  administered  to  the  incoming  students  who 
would  take  (^<GT  290'  to  determine  the  following  information:  self- 
reported  computer  experience,  self-reported  spreadsheet  experience, 
self-reported  (TBI  experience,  and  pre-course  spreadsheet  knowledge. 

The  Instructional  Method  Effectiveness  Measurement  for  the  CBI 
Experiment .  A  tool  was  needed  to  measure  the  relative  effectiveness  of 
the  computer-based  instruction  method  compared  to  the  in-class 
instruction  method.  As  n»ntioned  in  Chapter  I,  the  spreadsheet  unit  of 
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QMGT  290  traditionally  used  an  end-of-course  spreadsheet  assignment  as 
an  evaluation  instrument.  The  spreadsheet  assignment  was  customarily 
graded  on  a  "satisfactory-unsatisfactory"  basis.  To  minimize  disruption 
to  the  traditional  evaluation  method,  the  researcher  chose  to  use  this 
same  evaluation  to  compare  students’  performance  in  the  CBI  and  in-class 
groups.  To  transform  the  traditional  evaluation  scheme  from  a 
"satisfactory-unsatisfactory"  rating  to  a  scalar  measurement  instrument, 
an  evaluation  form  was  developed  (Appendix  B) .  Contents  of  the  form 
were  based  on  criteria  identified  by  the  literature  review  and  criteria 
obtained  from  experienced  (3<GT  290  instructors.  The  evaluation  form 
proposed  that  the  instructors  rate  each  student's  spreadsheet  assignment 
in  the  following  areas-,  achievement  of  project  goals,  creativity, 
documentation,  effective  use  of  the  spreadsheet's  capabilities,  and  the 
spreadsheet's  layout /organization.  A  draft  of  the  evaluation  form  was 
presented  to  the  instructors  who  would  teach  QM(5T  290  for 
recommendations  and  weightings  for  the  evaluation  elements.  Tie 
finalized  evaluation  form  was  presented  to  the  C3MGT  290  course  director 
for  approval.  The  finalized  evaluation  form  consisted  of  five  weighted 
elements,  and  a  total  score.  The  five  assignment  evaluation  elements 
and  their  weightings  are  provided  in  Table  2. 

The  instructors  agreed  to  rate  the  students  on  each  evaluation 
element  on  a  scale  of  1  (low)  to  5  (high).  The  raw  score  of  1  to  5  was 
multiplied  by  the  element  weight  to  obtain  a  final  score  for  each 
evaluation  element.  The  student’s  total  score  was  obtained  by  adding 
the  final  scores  from  each  assignment  element.  The  analysis  methods 
used  on  the  assignment  evaluation  results  are  described  under  Sub¬ 
objective  6. 
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TABLE  2 


ELEMEhTTS  FTOM  THE  QMGT  290  BTUDEOT  EVALUATION  FORM  AND 
WE  ASSOCIATED  ELEMENT  WEIGHTINGS 

Evaluation  Element  Element  Weight 

Achieved  project  goals  X  3 

Creativity  X  1 

Documentation  (on/off  disk)  X  1 

Effective  use  of  spreadsheet  capabilities  X  2 

Layout/Organization  X  2 

The  Instructional  Method  Efficiency  Measurement  for  the 
Experiment .  The  information  necessary  to  measure  instructional  method 
efficiency  included  the  literature  review  of  Sub-objective  1,  attendance 
records  of  the  students  who  volunteered  to  participate  in  the  in-class 
group,  and  responses  of  the  CBI  student  volunteers  on  a  post-course 
survey.  Findings  of  the  literature  review  and  the  in-class  attendance 
records  are  presented  in  Chapter  III.  The  post-course  survey  of  the  CBI 
students  (referred  to  as  the  post -QMGT  290  student  survey  for  the 
reminder  of  this  report)  asked  the  students  to  log  the  amount  of  time 
they  spent  using  the  CBI  program.  A  copy  of  the  post-OKJT  290  student 
survey  is  provided  at  Appendix  C.  The  analysis  methodology  of 
instructional  method  efficiency  is  addressed  under  Sub-objective  6. 

Threats  to  the  Experiment’s  Validity.  Each  of  the  eight  threats 
to  internal  validity  (10:115-117)  was  considered.  Characteristics  of 
the  students  that  volunteered  for  the  CBI  and  in-class  groups  were 
measured,  and  considered  in  the  final  analyses  and  conclusions.  The 
experiment  lasted  only  14  days  with  no  member  attrition:  thus  mortality, 
historical,  and  long-term  maturation  effects  were  controlled.  Inherent 
to  the  (CBI  group  was  the  freedom  to  choose  when,  where,  and  how  quickly 
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to  take  the  course  material .  This  provided  control  over  short-term 
maturation  effects  such  as  hunger,  tiredness,  or  boredom  (10:116).  The 
in-class  group  attended  traditionally  structured  classes,  and  therefore 
had  reduced  control  over  short-term  maturation  effects. 

Potential  bias  of  the  subjective  post-test  evaluations  was  an 
instrumentation  threat  to  validity.  The  effectiveness  measurement 
instrument  was  the  course  director  and  instructor  approved  assignment 
evaluation  form  (described  previously).  The  researcher  relied  on  the 
QMGT  290  instructors  to  remain  unbiased  in  their  evaluations  of  student 
assignments.  The  differences  between  the  instructors’  evaluation 
ratings  were  considered  in  the  final  analyses  and  conclusions. 

Availability  of  Computer  Resources  for  the  Experiment .  The  (TBI 
program  required  that  students  have  sufficient  access  to  computer 
resources,  specifically  the  computer  hardware  and  software.  Lack  of 
sufficient  resour'ces  could  have  had  a  detrimental  effect  on  measurement 
of  the  CBI's  effectiveness  and  efficiency.  To  determine  the 
availability  of  computer  resources  for  the  typical  School  of  Systems  and 
Logistics  student,  a  survey  of  students  who  had  previously  taken  QMGT 
290  was  created.  The  survey  asked  the  former  students  their  opinions  on 
the  availability  and  adequacy  of  the  School  of  Systems  and  Logistics' 
computer  resources.  The  survey  also  sought  information  regarding  the 
number  of  students  that  owned  home  computers  before  they  took  (^KT  290. 
and  the  number  of  students  who  used  CXiattro  Pro  for  QMGT  290.  The 
applicable  survey  results  are  summarized  and  analyzed  in  Chapter  III. 
Sub-obiective  4 

Create  and  verify  the  content  and  operation  of  a  CBI  program  that 
mirrors  the  course  requirements  of  the  spreadsheet  unit  of  (HJT  290. 
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Sub-objective  1  identified  the  relevant  CBI  design,  programming, 
and  human-computer  interface  features  desirable  in  a  computer-based 
instruction  program.  The  specific  course  requirements  for  the 
spreadsheet  unit  of  290  were  determined  under  Sub-objective  2.  The 

next  step  was  to  incorporate  each  identified  course  requirement  into  a 
comprehensive  and  effective  CBI  program,  using  applicable  CBI  design, 
programming  and  human-computer  interface  features.  Prototypes  of  the 
CBI  program  were  presented  to  a  panel  of  three  spreadsheet  novices,  two 
experienced  spreadsheet  users,  two  previous  QMGT  290  instructors,  the 
QMGT  290  course  director,  and  one  AFIT  faculty  member  not  directly 
involved  with  QMGT  290  for  content  and  operational  verification. 
Recommendations  from  these  sources  were  incorporated  into  the  final  CBI 
program.  A  floppy  disk  copy  of  the  CBI  is  maintained  at  the  AFIT  School 
of  Systems  and  Lxgistics/LSC,  Wright-Pat terson  Air  Force  Base,  Ohio. 

Sub-objective  5 

Conduct  the  CBI  Experiment. 

The  CBI  group  received  instruction  from  the  computer-based 
instruction  program  only.  Each  CBI  group  member  was  .assigned  a  package 
containing  computer  disks  and  the  CBI  program  installation  instructions. 
The  CBI  group  members  were  released  from  the  eight  classroom  hours  of 
the  spreadsheet  portion  of  QMGT  290.  The  in-class  group  attended  class 
in  the  traditional  manner.  Both  groups  were  allowed  to  ask  questions  of 
the  instructors.  All  CBI  software  related  questions  were  relay'=^  to  the 
researcher.  A  log  of  these  referrals  is  provided  at  Appendi.x  E). 
Identical  post-course  assignments  were  given  to  the  CBI  and  in-class 
groups.  A  copy  of  the  assignment  is  provided  at  Appendix  E.  Both 
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groups  were  allowed  to  ask  questions  of  the  course  instructor  to  clarify 
assignment  instructions.  The  experimental  analysis  techniques  are 
described  in  Sub-objective  6. 

Sub-objective  6 

Analyze  the  results,  and  state  the  findings  of  the  experiment  and 
the  literature  review. 

Selection  of  the  Data  Analysis  Methods  for  the  Experiment.  The 
(31GT  290  instructors  evaluated  each  of  their  in-class  and  CBI  students' 
post-course  assignments  on  the  evaluation  form  previously  described. 

A  Wilk-Shapiro  test  for  normality  (30:226-228)  was  performed  on 
the  pre-QMGT  290  student  survey  data,  the  pre-course  spreadsheet 
knowledge  test  scores,  and  the  post-course  spreadsheet  assignment 
performance  scores.  Based  on  the  results  of  the  Wilk-Shapiro  test,  the 
Spearman  Rank  Correlation  and  the  Kruskal-Wallis  test  were  chosen  as 
effective  nonparametric  statistical  analysis  methods  (21:965-969,960- 
985:  30:193-196). 

Analysis  Methods  Used  to  Determine  the  Effects  of  Student 
Characteristics  on  the  Experimental  Outcomes 

The  students  who  would  take  QfjT  290  during  the  research  period 
were  surveyed  prior  to  beginning  the  course  with  the  pre-QMGT  290 
student  survey.  The  Likert  scale  question  responses  were  normalized  to 
a  ten-point  scale.  For  yes-no  response  questions,  a  "yes”  response  was 
given  a  score  of  ten,  a  "no"  response  was  given  a  score  of  zero.  Blank 
responses  were  not  included  in  the  data  analysis.  The  scores  were 
grouped  by  content  into  three  measurement  categories:  self-reported 
spreadsheet  experience,  self-reported  computer  experience,  and  self- 
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reported  CBI  experience.  The  scores  were  analyzed  and  compared  in  three 
student  groups;  the  CBI  group,  the  in-class  group,  and  both  groups 

combined.  The  responses  to  questions  in  each  of  the  three  measxjrement 
categories  for  each  student  group  were  rank-correlated  with  a  pre-course 
spreadsheet  knowledge  test  score,  and  the  post -course  spreadsheet 
assignment  score.  The  question  responses  that  had  less  than  a  0.5  rank- 
correlation  were  extracted.  The  remaining  question  responses  were 
combined  to  form  a  composite  for  each  measurement  category  within  each 
student  group.  Table  3  lists  the  composites  for  each  measurement 
category  by  student  group.  Note  that  none  of  the  self-rated  CBI 
question  scores  had  a  correlation  of  over  0.5  with  the  pre  or  post¬ 
course  performance  test  scores. 


TABLE  3 

QUESTIONS  FROM  THE  PRE-QMGT  290  STUDENT  SURVEY  THAT  WERE  RETAINED  AND 
BECAME  COMPOSITES  FX)R  THE  MEASUREMENT  CATEGORIES 


Student  Groups 

Measurement  Category 

Combined  Grouos 

CBI  (5rouD 

In-class  Group 

Spreadsheet  Experience: 

III 

1,8,9  10 

III  8,9,10 

III  1.7. 8, 9. 10 

Computer  Experience: 

II 

1,8,9,13 

II  7.8,13 

II  7,8.9.13 

CBI  Experience: 

none 

none 

none 

(NOTE:  The  roman  numerals  II  and  III  represent  sections  II  and  III  of 
the  pre-QMGT  290  student  survey . ) 

The  composites  from  each  student  group  were  rank-correlated  with 
the  pre-course  test  and  post-course  assignment  scores  in  the  following 
ways: 
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-  self-rated  spreadsheet  experience  to  pre-course  spreadsheet 
knowledge  test  performance 

-  self-rated  computer  experience  to  pre-course  spreadsheet 
knowledge  test  performance 

-  self-rated  spreadsheet  experience  to  post-course  spreadsheet 
assignment  performance 

-  self-rated  computer  experience  to  post-course  spreadsheet 
assignment  performance 

The  findings  and  analysis  of  the  composite  correlations  are  presented  in 
Chapter  III. 

Analysis  Method  Used  to  Determine  Instructional  Method 
Effectiveness.  The  QMGT  290  instructors  evaluated  each  student's  post¬ 
course  spreadsheet  assignment  on  the  assignment  evaluation  form 
previously  described.  The  students  were  rated  from  1  (low)  to  5  (hi<jh) 
on  each  weighted  evaluation  element.  The  rating  for  each  evaluation 
element  was  multiplied  by  the  evaluation  element's  weight  to  determine 
the  evaluation  element's  final  score.  The  final  scores  from  each 
evaluation  element  were  summed  to  arrive  at  a  total  score  for  each 
student.  A  total  score  mean  was  calculated  for  the  students  in  the  CBI 
group  and  the  students  in  the  in-class  group  students.  A  median  test 
(30:197-199)  was  conducted  on  the  total  score  means  to  determine  if  the 
total  score  mean  of  the  CBI  group  was  different  from  the  total  score 
mean  of  the  in-class  group.  A  Kruskal-Wallis  test  was  then  conducted  to 
see  if  the  difference  in  the  means  was  statistically  significant.  Th>^ 
(2BI  and  in-class  groups  were  also  ccmpared  on  the  final  scores  tor  each 
of  the  evaluation  elements. 

To  determine  if  differences  between  the  instructors’  ratings  were 
meaningful,  a  separate  Kruskal-Wal 1  is  test  was  performed  on  the  data 
from  each  instructor. 
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Analysis  Method  Used  to  Determine  Instructional  Method  Efficiency. 

Two  methods  were  employed  to  determine  instructional  method  efficiency. 
The  first  method  was  a  literature  review  of  past  CBI  experiments  and 
studies.  Findings  of  the  literature  review  are  presented  in  Chapter 
III. 

A  second  indicator  of  instructional  method  efficiency  was  results 
of  the  previously  mentioned  post-QMGT  290  student  survey  (Appendix  C) . 
The  survey  asked  the  students  in  the  CBI  group  to  log  the  amount  of  time 
they  spent  using  the  CBI  program.  The  mean  time  was  calculated  and 
compared  to  the  amount  of  direct  instructional  time  the  in-class 
students  received. 

Analysis  Method  Used  to  Determine  Student  Attitudes  and  Opinions. 

Student  attitudes  and  opinions  were  studied  through  a  literature 
review,  and  through  responses  of  the  CBI  stt'd-  ts  on  the  post-QMGT  290 
student  survey  (Appendix  C) .  The  ="iivey  was  administered  to  the 
students  in  the  CBI  group  after  they  had  completed  the  CBI  course  and 
received  their  final  score  on  the  post-course  assignment.  The  survey 
asked  the  CBI  students'  to  rate  (on  a  1  to  7  Likert  scale)  their 
feelings  about  the  CBI  program  they  had  completed  in  the  following 
areas;  program  effectiveness,  ease  of  use,  the  author’s  writing  style, 
and  value  of  the  program  in  preparing  them  for  the  post-course 
assignment.  The  students  were  next  asked  to  philosophically  rate  thair 
feelings  about  CBI  a^  a  learning  tool.  The  students  were  also  acked  if 
other  courses  at  AFIT  should  be  taught  by  the  CBI  method,  and  what  type 
of  computer  user  they  would  recommend  take  the  CBI  course  that  they 
completed. 


17 


The  Likert  scale  responses  were  normalized  to  a  ten-point  scale. 


with  1  being  at  the  negative  end  of  the  scale.  The  normalized  mean 
score  from  each  question  was  calculated  and  used  as  an  indicator  of  the 
CBI  students'  attitudes  and  opinions  on  the  particular  question  area. 

No  attempt  was  made  to  compare  the  CBI  students'  attitudes  or  opinions 
to  the  attitudes  or  opinions  of  the  in-class  group. 
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Hi.  r’indingi.:  and  Results 


Chapter  Overview 

This  chapter  describes  findings  of  the  research  sub-objectives 
described  in  Chapter  II.  The  findings  of  each  sub-objective  are 
addressed  separately.  This  chapter  concludes  with  a  review  of  the  major 
findings. 

Sub-objective  1 

Develop  a  theoretical  knowledge  base  on  CBI  effectiveness  and 
efficiency,  and  examine  the  literature  on  CBI  course  design, 
programming,  and  human-computer  interface  features. 

Literature  Review  FindincE  on  Wavs  to  Measure  CBI  Effectiveness 
and  Efficiency.  The  literature  included  previously  conducted  CBI 
experiments,  studies,  and  meta-analyses.  Ihe  reviewed  articles  used  CBI 
effectiveness  and  efficiency  "measures"  to  assess  the  relative 
effectiveness  and  efficiency  of  the  computer-based  method  of 
instruction.  The  measures  that  the  literature  identified  as  most 
important  included  student  achievement,  student  attitudes,  course 
completion  statistics,  time  and  resource  efficiency,  and  the 
correlations  between  student  achievement  and  the  following:  computer 
experience,  subject  matter  knowledge,  disadvantaged  students, 
personality  type,  and  grade  level  (5:112-114;  18:532-537;  19:24-25; 
20:82) . 

Table  4  lists  the  measures  used  in  this  research  to  assess  the 
relative  effectiveness  and  efficiency  of  the  CBI  method.  Since  all 
students  were  required  to  complete  the  course,  course  completion 
statistics  were  not  useful.  Resource  efficiency  measurements  were 
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beyond  the  scope  of  this  research.  The  students  were  all  masters  degree 
candidates,  none  of  whcan  were  disadvantaged.  Determination  of 
personality  types  was  beyond  the  scope  of  this  research 

TABLE  4 

MEASURES  USED  IN  THIS  RESEARCH  TO  ASSESS  THE  RELATIVE 
EPreCTIVENESS  AND  EFFICIENCY  OF  THE  CBI  METHOD 


Instructional  method  time  efficiency 

Student  achievement 

Correlation  between  student  achievement  and  self-reported  prior 
spreadsheet  experience 

Correlation  between  student  achievement  and  self-reported  prior 
computer  experience 

Correlation  between  student  achievement  and  prior  subject  matter 
(spreadsheet)  )<now ledge 

Students'  self-reported  post-course  attitudes 


Literature  Review  Findings  on  the  Effectiveness  and  Efficiency  of 
CBI.  The  literature  review  findings  on  the  effectiveness  and  efficiency 
of  computer-based  instruction  are  addressed  under  Sub-objective  6.  along 
with  the  findings  of  this  research. 

Literature  Review  Findincrs  on  (TBI  Course  Design  and  Programming. 
Computer-based  instruction  in  and  of  itself  does  not  guarantee  quality 
instruction.  Quality  lies  in  the  content  and  design  of  the  program  that 
takes  advantage  of  the  medium's  potential  for  interactivity  (17:17). 
Quality  CBI  can  be  developed  by  combining  current  technology  with  sound 
educational  theory  (1:192).  The  following  paragraphs  outline  the 
literature  review  findings  on  designing  and  programming  CBI  courses, 
concluding  with  a  section  on  specific  guidelines  used  in  the  creation  of 
a  (TBI  program  for  this  research. 

General  CBI  Procrram  Design  Considerations.  CBI  program 
design  includes  programming  techniques,  styles,  approaches,  and  lesson 
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sequences.  An  effective  approach  to  the  issues  of  CBI  program  design  is 
the  "user-centered  approach"  (23:20).  In  this  approach,  the  students' 
capabilities,  the  course  tasks  and  goals,  and  information  requirements 
are  considered  early  in  the  process  of  course  design  (23:20). 

CBI  Program  Style.  To  a  certain  extent,  personal 
instruction  style  can  be  reflected  in  the  tone  of  the  GBI  lesson.  The 
author  must  choose  the  persona  or  personality  the  CBI  presents  to  the 
student.  Many  authors  choose  an  "invisible"  persona,  while  others 
employ  persona  characterized  by  humor,  enthusiasm,  or  concern  (27:64). 
The  author's  task  is  to  choose  the  persona  that  is  appropriate  for  the 
lesson  material,  and  that  appeals  to  the  students  in  such  a  way  as  to 
enhance  rather  than  distract  from  the  learning. 

Another  element  of  computer-based  instruction  program  style  is  the 
use  of  special  effects.  Special  eff.ects  can  add  emphasis,  capture,  and 
control  the  students'  attention  through  highlighting,  delays,  special 
characters,  graphic  displays,  animation,  or  scrolling  text  (27:64).  The 
task  of  the  CBI  author  is  to  ensure  special  effects  enhance,  not 
distract  from,  the  CBI  lesson. 

CBI  Lesson  Approaches.  OI  lessons  are  simply  tools  or 
frameworks  for  topic  presentation.  The  author  is  responsible  to  ensure 
appropriate  content  and  order  of  presentation.  CBI  has  the  advantage  of 
topic  customization,  but  requires  detailed  early  and  careful  preparation 
(15:105) . 

Computer-based  instruction  lesson  approaches  include  gaming,  drill 
and  practice,  simulation,  tutorial,  and  inductive  versus  deductive 
course  material  presentation  (27:58-60).  A  computer  lesson  using  a 
single  methodology  is  unlikely  to  be  successful  in  and  of  itself.  A  CBI 
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program  should  combine  lesson  approach  methods,  such  as  a  tutorial 
followed  by  drill,  then  a  test.  Multiple  lesson  approach  methods, 
however,  increase  program  length  and  complexity  beyond  the  scope  of  a 
single  lesson.  A  CBI  program  will  normally  be  more  successful  if  it  is 
based  on  a  series  of  combined  methodology  lessons  (1:147). 

Gaming  Lesson  Approach.  "The  advantage  of 
instructional  games  is  that  they  are  usually  more  engaging  than  other 
forms  of  instruction,  and  the  students  persist  in  using  them  longer" 
(1:150).  Gaming  is  especially  suitable  for  lower  grade  levels  but  can 
be  effective  with  adults  as  well  (27:58). 

Drill  and  Practice  Lesson  Approach .  The  dra 1 1  and 
practice  lesson  approach  includes  instruction  and  exercise  on  a  certain 
topic,  which  IS  usually  presented  in  a  simple  question  and  answer 
format.  The  computer  provides  practice  and  drills  on  a  topic,  and 
provides  the  students  immediate  feedback  on  their  performance  (5:110). 

At  the  conclusion  of  the  question  series,  the  computer  evaluates  the 
student's  performance  (15:104).  Drill  and  practice  is  normally  used  for 
basic  rather  than  advanced  material  (27:60). 

Simulation  Lesson  Approach.  An  advantage  of 
simulation  is  its  ability  to  imitate  and  simplify  reality  (1:150). 
Simulation  can  model  complex  systems  which  cannot  be  moved  into  the 
classroom  (5:110).  The  computer  can  store  great  amounts  of  material  in 
a  simulation  scenario,  presenting  a  series  of  problems  to  the  student. 
Simulations  support  the  comprehension  of  interrelated  facts  through 
interactive  decision  making  (15:105).  Simulations  involving  many 
students  are  sometimes  referred  to  as  gaming  (15:104). 
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Tutorial  Lesson  Approach.  A  CBI  tutorial  is  a 


complete  educational  process  that  contains  a  series  of  smaller  frame-by- 
frame  presentations  of  a  larger  topic  (15:104:  27:60).  The  frames  are 
presented  in  fixed,  though  flexible,  format  allowing  students  to  review 
or  branch  based  on  their  progress  (5:110:  15:104).  The  tutorial's 
strength  lies  in  the  presentation  of  text  and  graphics  enhanced  by  short 
question  and  einswer  evaluation  sessions  and  step-by-step  explanations 
(15:104-105) . 

Inductive  Versus  Deductive  Lesson  Approaches.  The 
author  chooses  to  design  the  CBI  lesson  flow  from  examples  to  rules 
(inductive)  or  from  rules  to  examples  (deductive).  Inductive  lesson 
design  is  good  for  discovery  type  lessons,  while  deductive  is  generally 
better  for  lecture  type  information  (2:3-9.47-53:  27:60) . 

Lesson  Design  Sequencing’.  Three  major  CBI  lesson  design 
sequencing  strategies  are  linear,  branching,  and  changing  (27:57). 

Linear  Sequencing.  Linear  sequencing  is  the  simplest 
and  most  commonly  used  design.  In  linear  sequencing,  all  students 
receive  the  same  material  in  the  same  sequence.  Strictly  linear 
sequencing  does  not  talce  full  advantage  of  CBI  flexibility  and  can  cause 
lX)redom  in  students  (27:57). 

Branching.  In  a  branching  lesson  sequence,  students 
can  be  directed  to  alternate  routes,  including  feedback  loops,  material 
review,  or  additional  practice  based  on  test  performance  or  the 
student's  request.  Branching  is  better  for  more  complex  subjects,  and 
IS  more  adaptable  to  student  and  instructor  styles  than  linear 
sequenc i ng  ( 27 : 57-58 ) . 
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Changing.  Changing  CBI  lessons  can  vary  with  each 


use.  Often  changing  CBI  as  generative,  consisting  of  banks  of  questions 
or  assignments  which  eire  randomly  or  systematically  presented  to  the 
student.  Changing  CBI  can  also  be  adaptive,  learning  from  the  students 
who  use  them,  auid  improving  future  instructional  material  (27:58). 

Literature  Review  Findings  on  l^jman-Computer  Interface  Features. 
Human-computer  interface  features  are  methods  through  which  the  computer 
and  student  communicate  and  interact. 

Hardware  Interface.  The  canputer  can  convey  information  to 
the  student  through  voice  synthesis,  equipment  panel  mock-ups,  a 
computer  monitor,  and  others.  The  student  can  communicate  with  the 
computer  using  a  joystick,  light  pen.  mouse,  keyboard,  track  ball, 
equipment  mockup,  touch-screen,  and  voice  recognition  (29:7). 

The  computer  resources  available  for  this  research  required 
keyboard  input,  and  computer  monitor  visual  output.  Although  no 
specific  instruction  was  provided,  the  students  could  also  use  a  mouse 
input  device. 

Software  Interface  Design.  Papazain  found  that  there  is 
cun'ently  no  widely  accepted  model  of  software  interface  design.  He 
stated  that  many  of  the  interface  guidelines  found  in  literature 
contradict  one  another.  There  is  little  consensus  concerning  criteria 
on  what  constitutes  a  "good"  interface  design.  The  only  thing  on  which 
most  human  factors  experts  agree  is  that  to  have  a  good  design,  the 
specific  purpose  of  the  (IBI  program  must  be  known  up  front.  Softwai'e 
interface  design  is  a  process  that  is  difficult  to  describe  precisely 
enough  to  be  useful  to  pseople  doing  specific  work,  and  general  enough  to 
remain  relevant  over  time,  or  for  more  than  one  specific  application 
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(23:2-3.20).  Moore  agreed  that  no  clear  interface  design  standards 
exist.  He  found  that  many  studies  have  been  conducted  and  much  written 
wath  respect  to  user  interface  guidelines,  but  there  remain  no  well- 
defined  standard  and  a  fair  amount  of  inconsistency  from  source  to 
source  (22:14). 

These  research  findings  underscore  the  difficulty  in  determining 
specific  CBI  interface  design  guidelines  that  are  universally  acceptable 
or  applicable. 

The  CBI  Program  Design  Used  for  This  Research.  Predicted 
characteristics  of  the  students  vrfio  were  going  to  ta)<e  CHTT  290  dui'ing 
the  research  period  were  collected  through  a  survey  of  students  who  had 
formerly  taken  QMGT  290.  The  predicted  characteristics  allowed  a  "user 
centered"  programming  approach  early  in  the  (TBI  design  process.  The 
program,  used  an  "invisible"  persona  to  enhance,  learning  by  focusing  the 
students'  attention  on  the  course  naaterial  rather  than  the  delivery 
method.  Special  effects  were  limited  to  highlighting  and  graphic 
displays.  More  elaborate  special  effects  had  the  undesirable  potential 
to  draw  the  students'  attention  away  from  the  course  material,  and 
towards  the  special  effect  itself. 

The  CBI  lesson  approach  included  a  ccmbination  of  tutorial  and 
drill  and  practice  methodologies  presented  in  a  deductive  manner.  The 
course  material  was  well-suited  for  frame-by-frame,  modularized 
presentation.  The  course  material  was  presented  in  three  modules,  each 
containing  three  lessons.  Each  lesson  included  10  to  15  frames  of 
course  material,  followed  l>y  a  five  question  dril 1-and-practice  test. 
Students  could  take  the  tests  at  any  time.  The  program  provided  instant 
feedback  after  each  test  question,  and  allowed  the  student  to 
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inunediately  review  relevant  course  material  or  to  continue  with  the 
test.  Scores  were  automatically  maintained  and  presented  at  the  end  of 
each  test,  and  upon  request.  The  student  had  the  option  to  accept  the 
final  score  or  retake  all  or  part  of  the  test  before  or  after  reviewing 
the  lesson  material.  The  program’s  modularity  allowed  for  ease  of 
updating,  modifying,  or  adding  additional  course  material. 

The  CBI  program  used  a  branching  lesson  sequence,  containing 
feedback  loops,  material  review,  and  additional  practice.  The  students 
had  full  control  over  the  program  and  could  take  lessons  in  the 
recommended  sequence  or  in  any  sequence  they  chose.  They  could  stop, 
start,  or  review  any  lesson  at  any  time. 

Examples  of  the  CBI  program's  lesson  frames  and  test  frames  are 
provided  in  Appendices  F  and  G.  A  computer  floppy  disk  copy  of  the  CBI 
program  is  currently  maintained  the  AFIT  School  of  Systems  and 
Logistics/LSC,  Wright-Patterson  Air  Force  Base,  Ohio. 

A  Discussion  of  the  Specific  CBI  Design  Guidelines  Used  in  the  CBI 
Program  Developed  for  This  Research.  Authors  of  computer-based 
instruction  design  have  various  and  scanetimes  incompatible 
recommendations.  One  technical  report  reviewed  63  computer-based 
instruction  related  articles,  and  generalized  CBI  design  into  57  "CBI 
Guidelines."  Of  the  57  guidelines.  38  had  some  disagreement  between 
authors,  and  8  had  conflicting  research  findings.  Forty-four  guidelines 
had  insufficient  research,  and  in  only  three  cases  was  there  agreement 
between  authors  and  sufficient  supporting  research  (31:37-41).  Many 
articles  presented  lists  of  generalized  CBI  guidelines,  principles, 
strategies,  considerations,  or  attributes  (3:1-13;  14:3-5,16-20:  22:14- 
36:  27:47-57;  28:  286-290;  31:37-40).  This  report  refers  to  the 
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identified  principles,  strategies,  considerations,  and  attributes  as  CBI 
guidelines.  Tbe  guidelines  considered  applicable  to  this  research  are 
presented  in  the  following  broad  categories;  general  guidelines,  screen 
design  guidelines,  menu  system  guidelines,  on-screen  text  and  graphics 
guidelines,  feedback  guidelines,  and  input  and  output  guidelines 

General  Guidelines.  Following  is  a  list  of  general  design 
guidelines  that  were  followed  in  the  CBI  program  for  this  research: 

1)  Make  the  CBI  simple,  easy  to  learn,  easy  to  use,  and  easy  to 
remember  (22:14;  27 : 286 ) . 

2)  Design  the  CBI  to  be  reliable,  standardized,  and  consistent 
throughout  (14:3;  22:14;  28:286), 

3)  Make  the  CBI  courteous  and  helpful  (22:14). 

4)  Allow  the  students  to  control  the  CBI.  including  rate  of 
presentation  (14:4,18;  22:15). 

5)  Make  the  f  nl  adaptable  to  the  students'  expertise/ level  of 
know’iT'  i  (14:4;  22:15;  28:286). 

6)  Minimize  the  memory  requirements  on  the  students  (14:3:  28: 
286)  . 

7)  Define  the  instructional  objectives  (27:47). 

8)  Present  questions  to  the  students  (31:37). 

9)  Keep  the  total  session  or  lesson  time  within  the  students' 
attention  span  (14:16). 

10)  Ensure  that  symbols  have  the  same  meanings  at  all  times 
(14:17) . 

11)  Provide  the  students  a  page-back  capability  to  review  previous 
material  (14:18). 

12)  Allow  the  students  to  easily  exit  lessons,  return  to  the  menu, 
and  exit  the  program  (14:18). 

13)  Create  the  program  in  a  modular  format  (14:18). 

Screen  Design  Guidelines.  The  computer  monitor  remains  the 
primary  means  for  the  computer  to  convey  information  to  the  student 
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(29:7) .  The  CBI  program  for  this  research  allowed  students  to 
interactively  choose  or  change  screen  colors  based  on  their  personal 
preference.  Following  is  a  list  of  other  identified  screen  design 
features  that  were  used  in  the  CBI  program; 

1)  Use  a  consistent  display  format  on  the  screens.  Keep  the 
location  of  information  categories  (e.g.  titles,  prompt  lines, 
error  messages,  help  messages,  menus,  etc.)  consistent  within 
the  program.  (14:17;  28;290). 

2)  Center  information  on  the  screen  (28:290). 

3)  Use  color,  boxing,  and  highlighting  rather  than  blinking  to 
focus  attention  on  important  segments  of  information  (14:16: 
17:131;  22:36). 

4)  Highlight  no  more  than  ten  percent  of  the  screen  at  one  time 
(14:16) . 

5)  Have  only  one  logically  connected  thought  per  screen  (28:290). 

6)  Use  titles  instead  of  numbers  to  identify  screens  (14:19). 

Menus  Systems  Guidelines.  Menus  are  recommended  for 
occasional  and  novice  users  (22:22).  Menus  are  an  important  feature  in 
making  computer-based  instruction  user-friendly.  They  are  the  "most 
powerful  GJIPs  (computer  user  interface  programs)  available."  and  should 
be  laden  with  user-friendly  features  (3:10).  Following  is  a  list  of  the 
identified  CBI  menu  features  used  in  the  CBI  program  for  this  research: 

1)  Ensure  that  menus  are  easy  to  learn  and  use  (22:36) .  Keep 
the  layout  simple  and  consistent  (3:12).  Make  the 
selections  self-explanatory,  explicit,  and  brief  (28:291) . 

2)  Have  three  to  nine  choices  on  each  main  and  sutmenu  (14:16). 

3)  Allow  the  student  to  make  a  menu  selection  with  upper  or  lower 
case  entries  (28:291). 

4)  Prevent  anything  from  happening,  (other  than  feedback)  if  an 
invalid  key  is  pressed  (3:12). 

5)  Provide  the  students  a  status  report  if  a  menu  operation  takes 
time  (3:12) . 
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6)  List  the  menu  choices  in  a  logical  order  with  the  most 
frequently  used  selections  at  the  top  (28:291). 

7)  List  menu  options  by  number  rather  than  letter,  where  order 
of  lessons  is  important  (14:19). 

8)  Limit  routing  menus  to  a  maximum  of  three  levels  (14:19). 

On-Screen  Text  and  Graphics  Guidelines.  For  this  study,  the 
computer  communicated  to  the  student  through  on-screen  text  and 
graphics.  The  following  list  of  desirable  on-screen  text  and  graphics 
features  was  considered  in  designing  the  CBI  program: 

1)  Limit  the  lines  of  text  per  screen,  preferably  to  no  more  than 
seven  (14:16). 

2)  Use  simple  syntax  in  the  active  voice  (14:16). 

3)  Justify  text  to  the  left,  numbers  to  the  right,  and  align 
decimal  points  (28:291). 

4)  Present  long  lists  in  some  logical  order  such  as  alphabetical, 
chronological,  or  numerical  (28:291). 

5)  Use  consistent  wording  convention  throughout  the  program. 

6)  Present  critical  information  at  the  beginning  of  a  message  or 
centered  on  the  screen  (14:17). 

7)  Have  no  more  them  three  or  four  consecutive  text  screens 
without  student  interactivity  (14:16). 

8)  Allow  no  more  than  five  seconds  for  text  euad  graphics  to  fill 
the  screen  (14:20). 

9)  Do  not  use  words  unique  to  the  ccmputer  field  (22:22) . 

Feedback  Guidelines.  "Research  indicates  that  feedback 
which  provides  information,  not  simply  immediate  feedback,  is  the  key  to 
performance  change:  .  .  .  informational  feedback  helps  the  student 
locate  the  error  and  construct  an  alternative  response"  (14:5).  Most 
authors  agree  that  CBI  programs  should  provide  the  student  with 
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specific,  informative  feedback  (1:148;  14:5.20;  22:15:  28:286;  31:37). 
Following  is  a  list  of  feedback  features  used  in  the  CBI  program: 

1)  Do  not  present  novel  or  entertaining  feedback  for  errors 
(31:38) . 

2)  Ensure  feedback  response  times  are  prompt  (22:14,15). 

3)  Keep  feedback  delay  constant  rather  than  variable  (14:17). 

4)  Provide  periodic  feedback  indicating  normal  operating  status 
if  the  student  must  stand  by  (14:20) . 

5)  Track  response  patterns  and  provide  feedback  on  areas  where 
the  student  nee^  remediation  (14:20). 

6)  Distinguish  feedback  from  other  text  through  use  of 
highlighting  techniques  (14:20). 

7)  Provide  students  a  performance  score  (14:20). 

8)  Pause  after  feedback  to  allow  time  for  consol idation  of  the 
newly  acquired  material  (14:20). 

Input-Qjtput  Guidelines.  Input  and  output  guidelines 
include  the  methods  and  conventions  used  by  the  student  and  the 
computer  to  effectively  and  efficiently  communicate.  The  following 
input-output  considerations  were  applied  to  the  C3I  program  for  this 
research : 

1)  Ensure  presentation  of  output  information  and  required  student 
input  -are  compatible.  Require  a  minimum  of  cognitive 
processing  to  understand  output  and  to  know  what  input  to 
provide  (14:4). 

2)  Conform  input  and  output  to  popular  stereotypes  (red  •=  stop, 
green  =  go)  (14:5,19). 

3)  Allow  students  to  correct  input  through  reentry  (22:15). 

4)  Do  not  use  multi /special  function  input  keys  other  the  ESC  key 
(22:36) . 

5)  Use  consistent  displays,  question  formats,  and  input/output 
requ i rements  (14:17:  22:36). 

6)  Allow  the  students  to  correct  and  recover  from  input  errors 
without  disrupting  the  lesson  sequence  (14.18). 
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7)  Require  students  to  respond  with  codes  only  when  necessary. 
Allow  students  to  type  in  the  first  letter  of  a  conunand  rather 
than  a  code  ("y"  =  yes  instead  of  "1"=  yes)  (14:19). 

8)  '7ive  the  students  more  than  one  chance  to  answer  (14:20) . 

9)  Allow  the  students  to  interact  frequently  (1:148). 

The  summarized  literature  review  findings  on  CBI  course  design, 
programming,  and  human-computer  interface  features  were  used  along  with 
the  (3<GT  290  course  requirements  (addressed  next  under  Sub-objective  2) 
to  cr'eate  the  Quattro  Pro  CBI  program  for  this  research. 

Sub-objective  2 

Identify  the  course  requirements  of  the  spreadsheet  unit  of  the 
Sschool  of  Systems  and  Logistics  course  QMGT  290. 

As  mentioned  in  Chapter  II,  this  research  sub-objective  was 
addressed  through  a  survey  of  the  QMGT  290  course  director  and 
instructors.  The  survey  interviews  identified  the  theoretical  knowledge 
and  mechanical  knowledge  elements,  listed  in  Table  5,  as  the  spreadsheet 
unit  course  requirements  for  CJ4GT  290.  All  of  the  identified 
requirements  were  included  in  the  computer-based  instruction  program. 

Sub-objective  3 

Determine  an  appropriate  experimental  design  to  use  for  this  study. 

Literature  Review  Findings  on  Experimental  Design.  ''There  was 
virtually  no  evidence  to  suggest  the  existence  of  a  relationship  between 
experimental  design  features  and  study  outcomes"  (4:3).  Kulik  agreed 
that  the  features  of  the  CBI  studies  and  experiments  were  not  related  to 
outcomes  (18:538;  19:25).  Reduced  experimental  design  bias,  however, 
enhances  the  be) ievabi I ity  of  CBI  research  findings.  The  Kulik 
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TABLE  5 


THE  SPREADSHEET  UNIT  OF  (H5T  290  COURSE  REQUIREMEhTTS  THAT  WERE 
IDENTIFIED  THROUGH  A  SURVEY  OF  THE  COURSE  DIRECTOR  AND  INSTRUCTORS 


Theoretical  Knowledge  Concepts: 

autcHnatic  recalculation 

consistency 

data  backup 

documentation 

logical  spreadsheet  design 

spreadsheet  uses/app 1 i cat ions 

Mechanical  Knowledge  Concepts: 

basic  math  functions 

-  add,  divide,  multiply,  subtract 
cell  blocks 

-  copy  blocks,  erase  blocks 

cells 

-  absolute  vs.  relative  referencing,  copy,  enter  data, 
erase 

columns 

-  change/set  width,  delete,  insert 
editing 

-  edit  cell  data 

files 

-  call  up  an  existing  file,  change  directory,  open  a 
new  file,  retrieve  an  existing  file  (replace 
current),  save  a  file 

graphs 

-  add  text,  choose  type,  create,  customize  series, 
modify  X  and  Y  axes,  name,  print,  specify  series, 
view 

other  operations 

-  basic  @  functions,  create  and  enter  formulas,  exit 
spreadsheet,  use  the  pull-down  menus 

print 

-  print  spreadsheet,  set  print  block 

rows 

-  delete,  insert 


secondary  school  meta-analysis  credited  stronger  CBI  experiment 
effectiveness  results  in  more  current  experiments  to  "more  appropriate 
use"  of  CBI  technology  rather  than  improved  design  features  (19:25). 
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The  Kulik  college  level  meta-analysis  found  the  only  variable  in 
experimental  design  affecting  study  outcome  was  the  use  of  the  same 
instructor  for  the  CBI  and  in-class  course.  The  studies  which  used 
different  instructors  for  cc;nput(H'  lMs<jd  and  in-cla.ss  sections  of  a 
course  had  more  clear  cut  examination  differeru:es  favoring  the  CBI 
sectlt'>ns.  Ibe  sl  ndies  whi<.h  use*!  a  srnyle  ii>;jtriici.or'  for'  Ixjth  classes 
produced  differences  that  were  less  pronounced.  Ihe  CBI  development 
requirements  of  outlining  objectives,  constructing  lessons,  and 
preparing  evaluation  materials  may  improve  the  instructors'  conventional 
teaching  assignments  (18:539) . 

The  CBI  program  for  this  study  was  designed  and  written  by  the 
-researcher.  The  instructors  were  involved  in  the  CBI  design  process 
only  to  the  extent  of  identifying  course  requirements. 

Selection  of  the  Experimental  Design  for  This  Research.  As 

mentioned  in  Chapter  II,  a  self-selected  experimental  group, 
quasiexperiment  design  was  used  due  to  that  design's  simplicity,  ease  of 
adaptation  to  the  current  QMGT  290  instructional  method,  and  the  course 
director's  requirement  that  the  students'  participation  in  the 
experiment  be  voluntary. 

Sub-objective  4 

(Treate  and  verify  the  content  and  operation  of  a  CBI  program  that 
mirrors  the  course  requirements  of  the  spreadsheet  unit  of  C2MGT  290. 

Determining  the  Feasibility  of  Testing  a  CBI  Program  at  AFIT 
School  of  Systems  and  Logistics.  Prior  to  arrival  of  the  students  who 
would  take  QMGT  290.  a  survey  of  students  who  had  taken  the  previous 
QMGT  290  course  offering  was  completed.  A  copy  of  the  survey  is 


33 


provided  in  Appendix  H.  Of  the  135  surveys  distributed.  89  were 
completed  and  returned;  a  66  percent  response  rate.  A  complete  list  of 
the  survey  findings  is  provided  in  Appendix  I.  Table  6  outlines  the 
survey  findings  that  helped  answer  the  following  questions  that  are 
related  to  the  feasibility  of  implementing  a  CBI  program  for  the 
spreadsheet  unit  of  QMGT  290: 

1)  Are  the  computer  resources  provided  by  the  AFIT  School  of 
Systems  and  Logistics  sufficient  to  implement  a  CBI  program? 

2)  How  many  CH3T  290  students  will  have  home  computers 
available  for  use  in  QMGT  290? 

3)  Is  Quattro  Pro  software  sufficiently  available  for  the  OtGT 
290  students? 

4)  Will  the  number  of  students  who  volunteer  to  participate  in 
the  CBI  experiment  be  sufficient  to  justify  a  self-selecte<l 
quasi-experiment  design? 

The  data  shows  that  75  percent  of  the  respondents  used  a  home 
computer  for  QMGT  290.  This  finding,  combined  with  the  finding  that  69 
percent  of  the  students  felt  that  the  AFIT  School  of  Systems  and 
Logistics  personal  computers  were  sufficient,  indicates  that  sufficient 
computer  hardware  resources  should  be  available  to  conduct  a  CBI  course. 

The  finding  that  88  percent  of  the  students  used  Quattro  Pro  for 
QMGT  290  (although  other  commercial  spreadsheet  packages  were 
permitted) ,  the  fact  that  the  CMTr  290  course  director  chose  Quattro  Pro 
as  the  in-class  spreadsheet  software,  and  the  fact  that  Quattro  Pro 
would  be  available  on  the  School  of  Systems  and  Logistics  personal 
computers,  support  the  choice  of  Quattro  Pro  as  the  CBI  program 
software . 

The  finding  that  32  percent  of  the  students  who  formerly  took  CMJT 
290  would  have  volunteered  to  take  the  course  by  CBI  indicated  that 
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TABUE  6 


RESULTS  OF  THE  FORMER  QMGT  290  BTUDEm’  SURVEY 


Number 

Percentage 

Used  home  computer  for  QMGT  290 

67 

75% 

Felt  AFIT  School  of  Systems  and 

Logistics  personal  computers 

are  sufficient 

61 

69% 

Used  Quattro  Pro  for  QMGT  290 

78 

88% 

Would  prefer  to  take  QMGT  290 

by  the  CBI  method 

28 

32% 

sufficient  numbers  of  the  students  who  would  take  QMGT  290  should 
volunteer  for  the  CBI  experiment.  Knowing  that  118  of  the  incoming 
students  were  eligible  for  the  CBI  course,  it  was  predicted  that 
approximately  one  third,  or  about  39  students  would  volunteer.  In 
actuality.  46  of  the  eligible  incoming  students  volunteered  to  take  the 
spreadsheet  portion  of  (^IGT  290  by  the  CBI  method. 

Creation  and  Verification  of  the  CBI  Program.  Sub-objective  1 
identified  the  relevant  CBI  design,  programming,  and  human-computer 
interface  features  desirable  in  a  computer-based  instruction  program. 
The  specific  course  requirements  for  the  spreadsheet  unit  of  QMGT  290 
were  determined  under  Sub-objective  2.  Each  course  requirement  was 
incorporated  into  Quattro  Pro  CBI  program,  using  the  design, 
programming,  and  human-computer  interface  features  identified  by  the 
literature  review.  As  mentioned  in  Chapter  II.  prototypes  of  the  CBI 
program  were  presented  to  a  panel  of  three  spreadsheet  novices,  two 
experienced  spreadsheet  users,  two  previous  (^IGT  290  instructors,  the 
QMGT  290  course  director,  and  one  AFIT  faculty  member  not  directly 
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involved  with  QMGT  290  for  content  and  operational  verification.  A 
floppy  disk  copy  of  the  CBI  program  is  maintained  at  the  AFIT  School  of 
Systems  and  Legist ics/LSC.  WrightHPatterson  Air  Force  Base,  Chio. 

Sub-objective  5 

Conduct  the  CBI  experiment. 

Each  CBI  group  student  was  assigned  a  package  containing  computer 
disks  and  the  CBI  program  installation  instructions.  The  CBI  group 
students  were  released  from  the  eight  classroom  hours  of  the  spreadsheet 
portion  of  QMGT  290.  The  in-class  group  students  attended  class  in  the 
traditional  manner.  Both  groups  were  allowed  to  ask  questions  of  the 
instructors  during  the  course.  All  CBI  software  related  questions  were 
relayed  to  the  researcher,  a  log  of  which  is  provided  at  Appendix  D. 
Identical  post-course  assignments  were  given  to  the  CBI  and  in-class 
groups.  A  copy  of  the  assignment  is  provided  at  Appendix  E.  Both 
groups  were  allowed  to  ask  questions  of  the  course  instructor  to  clarify 
the  assignment  instructions.  The  C^4GT  290  instructors  evaluated  their 
respective  students'  assignments.  Analysis  and  findings  from  the 
experiment  are  provided  under  Sub-objective  6. 

Sub-objective  6 

Analyze  the  results,  and  state  the  findings  of  the  experiment. 

Selection  of  the  Data  Analysis  Method  for  the  Experiment.  As 
stated  in  Chapter  II,  a  Wilk-Shapiro  test  for  normality  (30:226-228)  was 
performed  on  the  survey  data  from  the  pre-QMGT  290  student  survey  (a 
survey  of  the  students  who  would  take  C^KTT  290  during  the  research 
period),  the  pre-course  test  scores,  and  the  post-course  assignment 
scores.  The  data  were  not  normally  distributed  and  thus  did  not  meet 
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the  assumptions  required  for  parametric  statistics.  Hie  pre-QMGT  290 
survey  data  was  at  the  ordinal  level;  thus  the  median  was  the 
appropriate  measure  of  centrality  (10:88-91).  The  Spearman  Rank 
Correlation  and  the  Kruskal-Wall is  test  were  chosen  as  effective 
nonparametric  statistical  analysis  methods  (21:965-969.980-985;  30:193- 
196) . 

Presentation  of  Findings.  The  remainder  of  this  chapter  presents 

the  relevant  findings  of  this  research.  The  literature  review  findings 
in  each  subject  area  are  presented  first,  followed  immediately  by  the 
findings  from  this  research.  The  chapter  concludes  with  a  review  of  the 
major  findings. 

Student  Characteristics  That  Could  Confound  the  Experimental 
Findings  on  Instructional  Method  Effectiveness. 

Findings  from  the  Literature.  Although  not  always  in 
agreement,  the  reviewed  literature  indicated  that  student 
characteristics  such  as  age.  previous  education,  and  computer  experience 
may  have  an  influence  on  CBI  effectiveness  (20:81) .  Erwin  found  that 
students'  scholastic  ability  and  computer  attitude  influenced  the 
success  of  CBI  (12:221-233).  Student  personality  types  can  also  affect 
(TBI  effectiveness  (16:20).  Some  studies  found  that  CBI  achieved  the 
best  results  with  low  ability  stulents  (5:112),  while  other  studies 
found  CBI  to  be  more  effective  with  students  of  high  ability  level  and 
with  disadvantaged  students  (4:37-38).  The  Roblyer  meta-analysis  found 
no  statistically  significant  evidence  to  support  the  relationship 
between  student  characteristics  and  CBI  effectiveness  (26:54).  Kulik 
agreed  that  there  is  at  liest  only  a  small  correlation  l^etween  student 
aptitude  and  the  effectiveness  of  CBI  (18:536). 
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Findlncfs  from  This  Research.  For  this  study,  all  students 


possessed  at  least  a  BS  college  degree.  Selection  for  attendance  at 
AFIT  implied  a  proven  scholastic  ability  level.  The  most  sicrnificant 
confounding  variables  of  the  students  participating  in  this  research 
were  past  computer  and  subject  matter  (spreadsheet)  experience.  These 
variables  were  measured  on  a  survey  of  the  students  who  would  be  taking 
QMGT  290  during  the  research  period  (the  pre-OMGT  290  survey) .  A  copy 
of  the  survey  is  provided  at  Appendix  J. 

The  pre-OMGt  290  survey  was  administered  on  May  23.  1991.  Of  the 
118  qualified  respondents,  108  returned  a  completed  survey,  for  a  92 
percent  response  rate.  One  hundred  and  three  of  the  108  students  that 
returned  a  survey  elected  to  voluntarily  participate  in  the  experiment: 
46  for  the  CBI  group,  and  57  for  the  in-class  group.  The  15  students 
who  did  not  return  a  survey  or  volunteer  to  participate  in  the 
experiment  attended  class  in  the  traditional  method,  but  were  not 
represented  in  any  of  the  data  for  this  study.  (General  characteristics 
of  the  OBI  and  in-class  student  groups  are  provided  in  Appendix  K. 

Sub-objective  6.  in  Chapter  II,  described  the  method  used  to 
analyze  the  effects  of  the  identified  student  characteristics  on  the 
experiment.  The  results  of  the  composite  correlations  are  provided  in 
Table  7. 

Both  (self-reported)  prior  spreadsheet  experience  and  prior 
computer  experience  were  positively  correlated  with  pre-course 
spreadsheet  knowledge.  There  was  no  substantial  correlation  between 
prior  spreadsheet  or  computer  experience  and  performance  on  the  post¬ 
course  assignment. 
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TABLE  7 


RESULTS  OF  -mE  COMPOSITE  OORRELATIOf^  OF  THE  MEASUREMENT  CATEGORIES 
(STUDENT  CHARACTTERISriCS)  TO  'THE  PRE-CaJRSE  SPREADSHEET  KNOWLEIX3E  TEST 
SCORES  AND  THE  POST-COURSE  ASSIGNMENT  SCORES 


Student  Groups 


Combined  Groups 


CBI  Group 


In-class  Group 


Spreadsh . /Comp . 
Experience 


Spreadsh . /Comp . 
Experience 


Spreadsh . /Comp . 
Experience 


Pre-course 

knowledge  test:  .60  /.61  .61  /.61  .58  /.59 


Post-course 

assignment:  .10  /.29  .16  /.37  .00  /.ll 


Finding  1.  Both  (self-reported)  prior  spreadsheet 
experience  and  prior  computer  experience  were  effective  indicators  of 
pre-course  subject  matter  knowledge  for  both  the  CBI  and  in-class 
groups. 

Finding  2.  Neither  (self-reported)  prior  spreadsheet 
experience  or  prior  computer  experience  were  good  indicators  of 
performance  on  the  post-course  assignment  for  students  in  either  group. 

Finding  3.  The  students  who  self-rated  themselves 
less  experienced,  and  also  performed  poorer  on  the  pretest,  did  not 
perform  poorer  on  the  post-course  assignment  regardless  of  method  of 
instruction. 

Finding  4.  Students  with  very  different  levels  of 
incoming  experience  and  knowledge  performed  equally  well  on  the  post¬ 
course  assignment  after  receiving  either  the  in-class  or  computer-based 
methods  of  instruction. 
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Findings  on  Instructional  Method  Effectiveness. 


Findings  from  the  Literature.  The  Bums  meta-analysis  of  40 
prior  studies  found  that  CBI  programs  were  significantly  more  effective 
in  promoting  increeised  student  achievement  at  both  the  elementary  and 
secondary  instruction  level  (4:3).  The  Kulik  meta-analysis  of  secondary 
school  studies  found  that  in  39  of  the  48  studies,  students  from  the  CBI 
classes  scored  higher  on  examinations  (19:22).  It  also  stated  that  of 
25  statistically  significant  studies.  23  favored  higher  CBI  class 
achievement,  concluding  that  CBI  is  moderately  more  effective  than 
traditional  classroom  instruction  in  secondary  schools  (19:22-23). 

The  Kulik  meta-analysis  of  59  college  level  studies  found  that  CBI 
is  effective  at  the  college  level,  but  to  a  lesser  degree  than  in 
secondary  schools.  It  concluded  that  CBI  has  at  best  a  small  effect  on 
achievement  at  the  college  level  (18:536). 

At  the  secondary  school  level,  the  effectiveness  of  CBI  was 
especially  clear  in  studies  of  disadvantaged  and  low  aptitude  students, 
but  smaller  in  studies  of  talented  students  (19:25-26).  The  Kulik 
CO liege- level  meta-analysis  found  increased  CBI  achievement  in  highly 
achieving  students,  and  disadvantaged  students  (18:536).  Burns  found 
that  achievement  of  average  level  students  was  not  significantly 
enhanced  by  CBI  (4:3) . 

In  his  meta-analysis  of  82  CBI  related  dissertations  and  studies. 
Roblyer  found  no  statistically  significant  evidence  of  a  relationship 
between  student  ability  level  and  the  effectiveness  of  computer-ba.sed 
applications  (26:54) . 

Findings  from  This  Research.  Chapter  II  described  the 
methodology  used  to  measure  instructional  method  effectiveness.  Table 
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8  presents  the  students'  mean  scores  on  each  post-course  assignment 
evaluation  element,  and  the  mean  overall  ratings.  The  data  are  arranged 
by  each  instructor's  ratings  (instructor  1  and  instructor  2),  and  by  the 
instructors'  combined  ratings  (Both). 

TABLE  8 

THE  MEAN  RESULTS  FROM  THE  INSTRUCTORS'  EVALUATIONS  OF  THE  FOST-COURSE 
SPREADSHEET  ASSIGNMENTS  ARRANGED  BY  INSTRUCTOR  AND  STUDElfT  GROUP 


Student  Groups 


Combined  Groups 

CBI  Group 

In-class  Group 

Inst. 

1 

Inst. 

2 

Both 

Inst. 

Inst. 

1 

Inst. 

2 

Both 
Inst . 

Inst . 

1 

Inst. 

2 

Both 

Inst. 

Achieved 
project  goals: 

14.5 

12.6 

13.0 

14.8 

13.2 

13.3 

14.2 

12.3 

12.8 

Creativity: 

4.4 

3.3 

4.0 

4.4 

3.7 

4.2 

4.4 

3.5 

3,9 

Document at i on : 

4.3 

3.3 

•3.8 

4.3 

3.3 

4.0 

4.3 

3.3 

3.7 

Effective  use 
of  spreadsheet 
capabilities: 

8.9 

6.4 

7.4 

8.9 

6.1 

7.5 

8.9 

6.6 

7.4 

Layout  and 
organization: 

8.9 

6.1 

7.3 

9.0 

5.5 

7.3 

8.8 

6.5 

7.3 

OVERALL  RATING: 

40.9 

32.0 

35.3 

41.4 

31.9 

35.7 

40.5 

32.1 

34.9 

In  the  evaluations  made  by  both  course  instructors  (Both) ,  the  C!BI 


group  scored  as  high  or  higher  than  the  in-class  group  in  each  project 
element  and  in  the  overall  ratings.  A  median  test  (30:197-199)  showed 
52  percent  of  the  (TBI  students  and  40  percent  of  the  in-class  students 
scored  above  the  post -course  assignment  median  score.  A  Kruskal-Wal 1  is 
nonparametric  test  (21:965-969;  30:193-196)  concluded  that  there  is  not 
enough  evidence  (at  a  .05  alpha)  from  the  post-course  assignment  data  to 
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reject  the  hypothesis  that  the  CBI  and  in-class  students'  scores  are  the 
same. 

A  potential  confounding  variable  to  the  post-test  results  was  the 
diiference  in  ratings  between  the  instructors.  Table  8  shows  that 
instructor  2  consistently  rated  students  lower  than  instructor  1.  Table 
9  indicates  that  instructor  2  evaluated  fewer  of  the  CBI  students’ 
assignments  than  instructor  1. 

TABLE  9 

THE  NUMBER  OF  SUCEJiTS  THAT  EACH  INI^UCIOR  RATED  ARRANGED  BY 
iNsrmucroR  and  student  group 

Instructor 

Instructor  1 

Instructor  2 

TOTAL; 

This  difference  could  potentially  skew  the  instructors'  combined 
data  in  favor  of  the  CBI  group.  To  determine  whether  such  skewing 
occurred,  a  separate  Kruskal-Wal lis  test  was  performed  on  the  ratings 
from  each  of  the  instructors.  The  tests  revealed  that  there  was  no 
significant  difference  between  the  CBI  and  in-class  post -course 
assignment  ratings  for  either  of  the  two  instructors  (at  a  .05  alpha). 
There  was  no  significant  difference  between  the  CBI  and  in-class  group 
post-course  assignment  scores  with  the  instructors'  ratings  combined  or 
With  either  instructor's  individual  ratings.  Therefore,  there  was  no 
significant  difference  in  performance  on  the  post-course  assignment 
between  the  CBI  group  students  or  in-class  group  students.  The  CBI 
group  performed  as  well  as  the  in-class  group. 


Total  Students  CBI  Students  In-class  Students 

57  31  26 

46  15  31 

l03"  46  'W 


Finding  5.  Ihe  computer-based  mode  of  instruction  was 


as  effective  in  producing  student  learning  as  the  in-class  mode  of 
instruction. 

Findings  on  Instructional  Method  Efficiency. 

Findings  from  the  Literature.  The  reviewed  articles  agreed 

that  in  courses  for  which  a  valid  CBI  program  had  been  implemented  and 

efficiency  records  maintained,  the  students  learned  the  same  course 

material  in  less  time  using  CBI  than  by  traditional  classroom  means  of 

instruction.  The  Kulik  college  level  meta-analysis  stated  that  the  most 

dramatic  finding  was  related  to  instructional  time. 

In  every  study  in  which  computer-based  instruction  substituted  for 
conventional  teaching,  the  computer  did  its  job  quickly — on  the 
average  in  about  two-thirds  the  time  required  by  conventional 
teaching  methods.  It  is  clear  that  the  computer  can  function 
satisfactorily  in  college  courses  and  at  the  same  time  reduce  time 
spent  in  instruction.  (18:538) 

The  Bums  meta-analysis  concurred  that  all  studies  reporting  on 
efficiency  showed  it  took  less  time  for  students  to  learn  the  same 
course  material  through  CBI  than  through  conventional  instruction 
methods  (4:35). 

Findings  from  This  Research.  As  described  in  Chapter  II. 
a  survey  of  the  CBI  group  students  (administered  after  they  had 
completed  the  CBI  course  and  received  their  scores  from  the  post-course 
assignment)  eisked  them  to  log  the  amount  of  time  they  spent  using  the 
CBI  program.  A  copy  of  the  survey  is  provided  in  Appendix  C.  The  CBI 
students  completed  and  returned  41  of  the  46  surveys,  for  an  89  percent 
response  rate.  The  mean  time  that  the  students  reportedly  spent  using 
the  CBI  program  was  160  minutes  (2  hours  and  40  minutes) .  The  in-class 
students  were  each  required  to  attend  480  minutes  (8  hours)  of  direct 
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instruction  in  the  computer  classroom.  This  data  indicates  that  the  in- 
class  students  were  required  to  receive  three  times  more  instructional 
time  than  was  used  by  the  average  CBI  group  student  to  co\"^"  siiiij.iar 
course  material . 

Finding  6.  The  computer-based  mode  of  i.o^cruction 
produced  more  efficient  learning  than  the  in-class  mode.  The  students 
taking  the  course  by  CBI  used  substantially  less  time  than  the  in-class 
students  to  learn  the  same  course  material . 

Findings  on  Student  Attitudes  and  Opinions. 

Findings  from  the  Literature.  Students'  attitudes  towarxi 
subject  matter  were  difficult  to  measure  due  to  the  lack  of  studies, 
varying  rating  methods,  and. differing  definitions  of  a  "positive 
attitude."  Chan  found  that  80  percent  of  the  elementaiy  school  teachers 
who  responded  felt  CBI  improved  students'  attitudes  (5:112).  The  Kulik 
secondary  school  meta-analysis  stated  that  most  studies  reporting  on 
student  attitudes  find  CBI  students  have  more  positive  attitudes  toward 
the  study  material  (19:24).  The  Roblyer  meta-analysis  agreed  that  (TBI 
students  tended  to  have  more  positive  attitudes,  but  found  few  studies 
to  support  that  finding  (26:55),  In  the  Kulik  college  level  meta- 
analysis.  only  11  of  59  studies  reported  on  student  attitudes.  The  CBI 
students'  attitudes  were  higher  in  eight,  but  statistically  sicfnificant 
in  only  four  of  the  studies  (18:53). 

Findings  from  This  Research.  As  mentioned  in  Chapter  II.  the 
post-OMGT  290  student  survey  was  used  to  solicit  the  CBI  students 
attitudes  and  opinions  on  the  CBI  program  they  completed,  and  on  CBI  in 
general.  The  Likert  scaled  responses  were  normalized  to  a  1  (low)  to  10 
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(high)  point  scale.  Table  10  lists  the  eireas  that  the  students  rated, 
and  the  mean  scores  of  the  students'  responses. 


As  Table  10  indicates,  the  CBI  students  had  a  strong,  positive 
attitude  toward  CBI  in  all  of  the  question  areas. 

Finding  7.  After  the  course,  the  CBI  students 
demonstrated  a  strong  positive  attitude  about  (BI,  and  the  specific  <BI 
Cf.'iurse  they  completed. 

TABLE  10 

MEAN  SCORES  OF  THE  NORMALIZED  RESPONSES  OF  THE  CBI  STUDENTS  TO  THE 
ATTITUDE  AND  OPINION  QUESTIONS  ON  THE  POST-QMGT  290  STUDENT  SURVEY 


(Question  Area 


Means  of  the  Responses 
(l=low,  10  =  high) 


Effectiveness  of  the  (BI  Program  7.9 

Ease  of  Use  of  the  CBI  Program  8.3 

The  author's  writing  style  8.5 

Value  of  the  CBI  pi ogram  in  helping  do  the 

post-course  assigriment  6.6 

Feelings  about  CBI  as  a  learning  tool  8.5 


The  lowest  attitudinal  rating  was  the  students’  opinions  on  the 
value  of  the  CBI  progr'am  in  helping  them  do  the  post-course  assignment. 
This  slightly  lower  rating  could  be  attributable  to  the  fact  that  the 
post-course  assignment  required  the  students  to  use  a  spreadsheet 
function  that  was  not  specifically  addressed  in  the  CBI  program. 
(Although  (Quattro  Pro  [at]  functions  in  general  were  determined  to 
be  a  course  requirement,  no  specific  @  functions  were  identified.  Some 
Quattro  Pro  @  functions  were  addressed  by  the  (BI,  but  not  the  specific 
@IF  function  that  was  used  in  the  post-course  assignment.  The  @IF 
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function  was  addressed  in  the  in-class  instruction.)  This  slight 
difference  in  course  content  between  the  in-class  instruction  and  the 
computer-based  instmction  probably  did  not  have  a  major  impact  on  the 
findings  of  this  study.  More  significant  differences  could  affect  the 
researcher’s  ability  to  precisely  compare  the  instructional  methods. 

The  survey  also  asked  for  other  opinions  from  the  students.  Of 
the  41  respondents.  36  {or  88  percent)  felt  that  if  the  course  material 
were  appropriate,  other  courses  at  AFIT  should  be  taught  by  the  CBI 
method.  When  asked  to  indicate  the  types  of  computer  users  they  would 
recommend  to  take  the  CBI  course  they  had  just  completed,  24  (59 
percent)  of  the  students  recommended  inexperienced  computer  users.  36 
(89  percent)  recommended  moderately  experienced  useis.  and  22  (54 
percent)  recommended  very  experienced  computer  users.  This  response  is 
interesting  in  view  of  the  fact  that  students  of  very  different  levels 
of  experience  and  knowledge  performed  equally  well  on  the  post-course 
assignment.  From  the  students'  written  comments,  some  felt  that 
ine:<per fenced  users  needed  human  instructor  interaction  to  get  started 
with  the  CBI  program  and  the  new  spreadsheet  package.  This  feeling 
could  be  attributable  to  the  initial  difficulty  some  students  faced  in 
loading  and  starting  the  CBI  program.  Although  the  CBI  program  start-up 
procedures  were  thoroughly  tested  by  the  researcher  and  three  other  AFIT 
students  just  one  week  prior  to  the  beginning  of  the  CBI  course, 
unforseen  problems  occurred  that  caused  confusion  among  some  f  the  CBI 
students.  One  such  problem  was  a  major  change  in  the  default  screen 
layout  of  the  upgraded  version  of  Quattro  Pro  that  some  students 
purchased.  The  CBI  program  was  written  in  the  macro  language  of  Quattro 
Pro  version  2.0.  Many  of  the  students  purchased  the  newer  Quattro  Pro 
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version  3.0.  Although  the  macro  language  was  totally  compatible,  a 
change  to  the  default  screen  layout  of  version  3.0  (a  "what  you  see  is 
what  you  get"  environment)  caused  confusion  among  students.  The  problem 
itself  was  easily  corrected;  communicating  the  correction  to  each 
student  in  a  timely  fashion  was  initially  difficult.  This  problem 
demonstrated  a  potential  limitation  in  CBI  programs  that  are  designed 
for  specific  versions  of  software. 

Finding  8.  Software  upgrades  or  changes  could 
significantly  affect  the  usability  and/or  quality  of  a  CBI  program. 

A  log  of  other  reported  problems  that  the  CBI  students  encountered 
is  provided  in  Appendix  D. 

A  little  over  half  of  the  respondents  did  not  recommend  the  CBI 
program  for  very  exp»erienced  computer  users.  One  purpose  of  the 
spreadsheet  unit  of  QMGT  290  was  to  provide  the  students  the  knowledge 
and  ability  to  use  automated  spreadsheets  in  their  graduate  program. 
Since  the  more  experienced  computer  users  might  have  already  possessed 
that  ability,  they  may  have  felt  that  the  CBI  program  was  simply  a 
review,  and  thus  not  necessary. 

Other  Findings  from  This  Research. 

A  Comparison  of  the  Scores  of  the  I>~i-class  Students  to  the 
CBI  Students  on  the  Pre-course  Spreadsheet  Knowledge  Test .  A  median 
test  on  the  results  of  the  pre-course  spreadsheet  knowledge  test 
revealed  that  55  percent  of  the  CBI  students,  aixl  40  percent  of  the  in- 
class  students  scored  higher  than  the  median  score.  A  Kruskal-Wal 1  is 
test  found  that  there  was  enough  evidence  from  the  data  to  reject  the 
hypothesis  that  the  scores  were  the  same  (at  a  .05  alpha). 
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Finding  9.  The  students  vrfio  volunteered  to  take  the 


course  by  the  computer-based  mode  scored  higher  on  the  pre-course 
knowledge  test  than  the  students  who  volunteered  to  participate  in  the 
experiment  by  the  traditional  mode. 

Pre-course  Spreadsheet  Knowledge  Compared  to  Post-course 
Spreadsheet  Assignment  Performance.  A  Spearman  Rank  Correlation  test 
found  no  correlation  (-0.03)  between  the  pre- -course  spreadsheet 
knowledge  test  and  the  post-course  spreadsheet  assignment  scores  with 
the  CBI  and  in-class  groups  combined.  The  CBI  group's  scores  had  a 
small,  positive  (0.41)  correlation,  while  the  in-class  group's  scores 
h<ad  a  weaker  (0.21)  correlation.  There  is  no  evidence  to  indicate  that 
students  scores  on  the  pre-course  knowledge  test  were  strongly 
cotT'elated  with  their  scores  on  the  post-course  assignment. 

A  Comparison  of  the  Required  Resubmission  Rates  of  the  (rST 
Group  and  the  In-class  Group.  Students  who  did  not  meet  minimum 
achievement  standards  were  required  to  correct  and  resulanit  their  post¬ 
course  assignment.  Table  11  shows  the  required  resubmissions  airang-sd 
by  instructor  and  student  group. 

(Overall  (TOTAL),  the  CBI  students  had  a  lower  resubmission  rate 
than  the  in-class  group.  By  percentages,  instructor  1  required  fewer 
<CBI  group  resubmissions.  while  instructor  2  required  fewer  in-class 
group  resubmissions.  There  is  not  sufficient  data  to  determine  whether 
there  is  a  significant  difference  in  the  required  resubmissions  between 
the  student  groups. 

Literature  Review  Findings  on  CBI  Course  Completion .  Chan 
fcxind  that  at  the  elementary  school  level,  slower  learning  students  were 
more  likely  to  complete  the  CBI.  The  learning  disadvantaged  students 
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TABI.E  11 


THE  NUMBETB  OF  STUDENTS  REOJIRED  TO  RESUBMIT  THEIR 
POST-OOURSE  AISSIONMENI  FROJECTS 


Student  Groups 


Both  Groups 


CSI  Group  In-class  Group 


(%  tot) 


(%  CBI)  (%  in-class) 


Instructor  1  8  (14.0%) 

Instructor  2  8  (17.4%) 


2  (6.9%)  6  (21.4%) 

3  (20.0%)  5  (16.1%) 


TOTAL:  16  (15.5%) 


5  (11.4%)  11  (18.6%) 


Note;  The  percentages  (other  than  TOTAL)  are  percentages  of  the  total 
number  of  students  rated  by  the  particular  instructor  in  the  particular 
student  group.  Example:  For  Instructor  1,  the  6.9  %  (under  the  CBI 
Group)  represents  the  percentage  of  the  total  number  of  CBI  student 
assignments  that  instructor  1  evaluated  and  required  to  be  resubmitted. 


preferred  ccmputer  interaction  over  human  interaction  due  to  the  lack  of 
negative  feedback  and  infinite  patienc3  of  the  machihe.-  Often  computer- 
based  lessons  revitalized  their  interest  (5:112-113).  The  Kulik  college 
level  meta-analysis  found  13  studies  reporting  on  CBI  course  completion. 
In  seven  of  the  studies  withdrawal  rate  was  higher  in  the  CBI  section, 
and  in  six  studies  withdrawal  rate  was  higher  in  the  conventional 
section  (18:536).  The  article  found  no  significant  difference  in  course 
completion  between  (TBI  and  conventional  teaching  methods  (18:536).  The 
Hoffman  personality  type  article  found  that  all  sensing  types  tended  to 
complete  the  CBI  sooner  than  the  intuitive  types.  The  extraverted 
perceptive  types  overwhelmingly  tended  to  drop  out  of  the  CBI  program 
(16:21).  The  article  found  that  the  CBI  course  withdrawal  rate  is 
related  to  student  characteristics  rather  than  course  delivery  method 
(16:20) . 
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A  Review  of  the  Major  Findings. 


For  the  convenience  of  the  reader,  this  section  repeats  the  major 
findings  of  this  research  project.  All  findings  should  be  interpreted 
within  the  scope  of  this  study. 

Finding  1.  Both  (self-reported)  prior  spreadsheet  experience  and 
prior  computer  experience  were  effective  indicators  of  pre-course 
subject  matter  knowledge  for  both  the  CBI  and  in-class  groups. 

Finding  2.  Neither  (self-reported)  prior  spreadsheet  experience 
or  prior  computer  experience  were  good  indicators  of  performance  on  the 
post-course  assignment  for  students  in  either  group. 

Finding. 3.  The  students  who  self-rated  themselves  less 
experienced  and  also  performed  poorer  on  the  pretest,  did  not  perform 
poorer  on  the  post-course  assignment  regardless  of  the  method  of 
instruction. 

Finding  4.  Students  with  very  different  levels  of  incoming 
experience  and  knowledge  performed  equally  well  on  the  post-course 
assiignment  after  receiving  either  the  in-class  or  computer-based  methods 
of  instruction. 

Finding  5.  The  computer-based  mode  of  instruction  was  as 
effective  in  producing  student  learning  as  the  in-class  mode  of 
instruction. 

Finding  6.  The  computer-based  mode  of  instruction  produced  more 
efficient  learning  than  the  in-class  mode.  The  students  taking  the 
course  by  CBI  used  substantially  less  time  than  the  in-class  students  to 
learn  the  same  course  material . 
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Finding  7.  After  the  course,  the  CBI  students  demonstrated  a 


strong  positive  attitude  about  CBI,  and  the  specific  CBI  course  they 
completed. 

Findincr  8.  Software  upgrades  or  changes  could  significantly 
affect  the  usedJility  and/or  quality  of  a  CBI  program. 

Finding  9.  The  students  vrfio  volunteered  to  take  the  course  by  the 
ccanputer-based  mode  scored  higher  on  the  pre-course  knowledge  test  than 
the  students  who  volunteere<i  to  participate  in  the  experiment  by  the 
traditional  mode. 
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IV.  Conclusions  and  Recommendations 


Chapter  Overview 

This  chapter  presents  the  conclusions  drawn  from  the  research  and 
offers  recommendations  for  future  computer-based  experiments  or  studies. 
The  conclusions  should  be  interpreted  within  the  scope  of  this  study. 

All  of  the  conclusions  are  presented,  first,  followed  by  the 
recommendations . 

Conclusion  1 

The  ccmputer-based  method  of  instruction  was  significantly  more 
efficient  and  produced  learning  effects  similar  to  the  in-class  method 
of  instruction  for  all  students.  The  AFIT  School  of  Systems  and 
Logistics  may  realize  important  time  savings  by  using  the  CBI  mode  for 
this  or  similar  course  material. 

Conclusion  2 

Since  the  student  characteristics  at  the  beginning  of  the  course 
were  offset  by  learning  from  either  instructional  mode,  and  since  the 
CBI  was  as  effective  as  the  in-class  mode,  faculty  can  safely  assign 
students  to  CBI  In  the  belief  that  the  students  will  leam  as 
effectively  as  they  would  leam  in  the  classroom,  regardless  of  the 
students'  initial  knowledge  or  experience  level. 

Conclusion  3 

Self-reported  subject  matter  experience  was  a  reasonably  effective 
indicator  of  actual  pre-course  subject  matter  knowledge  among  the 
students.  Faculty  could  use  the  students'  self-reported  experience  to 
achieve  better  balanced  classes,  and/or  to  determine  what  level  (e.g.. 
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novice,  intennediate,  experienced)  of  a  modularized  CBI  program  to 
assign  to  the  students. 

Conclusion  4 

Since  prior  experience  and  knowledge  were  not  closely  correlated 
with  the  post-course  assignment  performance,  it  can  be  inferred  that  the 
course  presented  in  either  instructional  mode  made  some  type  of 
difference  in  the  students'  knowledge  level. 

Conclusion  5 

The  students  who  performed  poorly  on  the  pre-course  knowledge 
test,  and  who  reported  themselves  low  in  experience  and  knowledge,  were 
not  identifiably  weaker  after  receiving  instruction  from  either 
instructional  mode.  Both  modes  had  a  "leveling"  effect,  perhaps  giving 
the  most  benefit  to  the  initially  "weaker"  students. 

Conclusion  6 

If  the  use  of  CBI  is  voluntary,  the  students  who  consider 
themselves  more  experienced  and  who  are  (initially)  more  knowledgeable 
are  more  likely  to  volunteer. 

Conclusion  7 

With  very  few  exceptions,  the  students  who  volunteer  for  CBI  can 
be  expected  to  feel  positive  about  the  use  of  CBI  at  the  end  of  their 
learning,  regardless  of  their  initial  experience  or  knowledge  levels. 

Recommendation  1 

The  AFIT  School  of  Systems  and  Logistics  should  consider  producing 
CBI  programs  for  other  courses  that  have  common  variables  and  are 
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similar  in  course  content  to  the  spreadsheet  unit  of  QMGT  290.  This 
research  showed  that  CBI  can  be  as  effective  and  more  efficient  than  in- 
class  instruction  in  teaching  basic  spreadsheet  functions.  A  logical 
next  step  is  to  investigate  CBI's  effectiveness  and  efficiency  with 
other  units  of  {^*37  290.  specifically  the  units  on  introduction  to 
databases  and  introduction  to  word  processors.  It  is  possible  that  much 
of  QMGT  290  could  be  taught  by  CBI.  Since  computers  are  becoming 
increasingly  commonplace,  a  CBI  course  could  be  distributed  and  taken  by 
students  before  they  arrive  at  AFTT. 

Recommendation  2 

Although  CBI  is  an  effective  instructional  mode,  it  should  not  be 
assumed  that  CBI  is  desirable,  suitable,  or  feasible  for  all  types  of 
course  material.  A  thorough  cost-benefits  analysis  should  be  pierformed 
to  determine  the  desirability,  suitability,  and  feasibility  of  a  CBI 
program  for  a  particular  course.  Creating  a  CBI  program  requires  a 
large  amount  of  planning,  preparing,  programming,  testing,  aixi 
resourcing.  Purchasing  "off-the-shelf"  CBI  could  require  a  large 
capital  investment.  Consideration  should  be  given  to  the  resources  that 
are  required  to  implement  a  CBI  program,  including  computer  hardware, 
computer  software,  programmer (s) ,  facilities,  and  software  distribution 
media  (floppy  disks,  installation  guides).  CBI  should  be  considered 
practical  only  after  a  cost-benefits  analysis  reveals  that  the  potential 
benefits  of  CBI  (improved  learning  efficiency,  standardization, 
convenience,  flexibility,  and  seif  paced  learning)  outweigh  the  total 
qualitative  and  quantitative  costs. 
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Recommendation  3 


CBI  should  not  be  targeted  towards  a  specific  version  of  canputer 
software.  The  CXiattro  Pro  software  package  used  for  this  research  was 
upgraded  by  the  manufacturer  twice  in  less  than  one  year.  The  changes 
caused  minor  problems  for  the  students  who  had  purchased  the  latest 
version.  Also,  computer  software  is  becoming  increasingly  homogeneous. 
The  specific  name  brand  of  software  used  in  a  course  could  change 
between  course  offerings,  potentially  rendering  a  CBI  program  obsolete. 
The  hazards  of  software  changes  and  obsolescence  should  be  considered 
prior  to  writing  a  CBI  program. 

Recommendation  4 

Another  factor  contributing  to  CBI  obsolescence  is  changes  in 
course  material  between  or  during  cotjrse  offerings.  Small  changes  in 
the  course  material  could  add  a  significant  amount  of  time  to  the 
programming  task  of  updating  earlier  CBI  lessons.  (Thanges  made  j'lst 
before  or  after  the  course  begins  would  be  difficult  to  include  in  the 
CBI  program.  The  computer-based  method  of  instruction  should  be 
considered  only  for  those  courses  that  change  infrequently. 

Recommendation  5 

The  CBI  author  should  ensure  that  the  CBI  program  is  flexible 
enough  to  accommodate  the  novice  user  while  allowing  the  more 
experienced  or  knowledgeable  user  to  rapidly  complete  the  course 
material.  A  recommended  approach  is  to  modularize  the  CBI  program  at 
the  student  ability  levels.  An  author  could  divide  the  CBI  program  into 
novice,  intermediate,  and  advanced  modules,  and  assign  the  modules  based 
on  the  students'  self-reported  experience  levels. 
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Closing  Remarks 


It  is  clear  that  technological  advances  will  play  an  increasingly 
large  role  in  the  future  of  education  and  training.  All  educational 
organizations,  including  AFIT,  must  continue  to  investigate  the 
potential  of  state-of-the-art  educational  technologies  such  as 
interactive  video  disk  (IVD),  robotics,  telecommunications,  expert 
systems,  simulations,  voice  recognition,  and  neural  systems..  As  this 
research  demonstrated,  one  educational  technology  with  great  potential 
is  computer-based  instruction. 
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Appendix  A:  The  Course  Director  and  Course  Instructor 
Survey 


Survey  Instructions 


1.  Based  on  your  experience  and  expertise,  you  are  requested  to 

participate  in  a  research  project.  Your  responses  to  the  items  in  this 
survey  will  help  determine  specific  requirements  for  a  computer  based 
version  of  the  automated  spreadsheet  unit  of  290,  Introduction  to 

AFIT  Computer  Systems. 

2.  The  researcher  will  personally  administer  this  survey. 

3.  Open  responses  are  encouraged.  If  you  feel  the  survey  does  not 
adequately  address  any  issues,  please  feel  free  to  comment  or  add 
additional  information. 

4.  For  the  purpose  of  this  survey.  NON- -MECHANICAL  concepts  are 
techniques,  ideas,  functions,  or  operations  that  you  believe  are 
important  to  any  spreadsheet  usage  regardless  of  the  internal 
capabilities  of  the  automated  spreadsheet  package.  MECHANICAL  concepts 
are  techniques,  ideas,  functions,  or  operations  directly  related  to  the 
internal  capabilities  of  an  autcanated  spreadsheet  package. 

5.  This  survey  begins  with  NON-MECHANICAL  concepts,  followed  by 
generalized  MEQTANICAL  concepts.  Each  MECHANICAL  concept  is  then  broken 
down  into  specific  spreadsheet  functions.  The  survey  concludes  with  a 
prototype  standardized  project  evaluation  form  provided  for  your 
recommendations  and  comments. 

6.  Thank  you  for  your  assistance  and  support  of  this  research  project. 


Russell  A.  Greene 
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NON-MECHANICAL  CONCEPTS 


--  -  1 

AFTT  LAN  considerations 

- n 

- ^ 

AFTT  spreadsheet  software  availability 

DOS 

1 

commercial  spreadsheet  availability 

data  backup 

1 

1  definition 

1  documentation 

formulas 

1 

hard/floppy  disk  file  mana^^ement 

hardware  requirements 

keyboard  layout/functions 

logical  design  techniques 

mouse  functions 

1 

physical  layout/organization 

.^^1  1 

printer  functions 

1 

software  copyrights 

software  loading 

software  requirements 

spreadsheet  uses/applications 
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basic  math  functions 

21 

cell  blocks 

1 

22 

ceUs 

HI 

SI 

1 

23 

columns 

1 

24 

database 

! 

25 

edit 

“ 

26 

file 

27 

graph 

28 

macros 

29 

menus 

30 

other  advanced  operations 

31 

other  general  operations 

32 

print 

33 

rows 

34 

setup  options 

35 

windows 
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CELL  BLOCKS 


40 

41 

42 

43 

44 

45 


change/set  alignment 


copy  blocks 


erase  blocks 


fill  blocks  (numbers) 


move  blocks 


name  blocks 


CELLS 


46 

copy  cells 

47 

erase  cells 

48 

modify  cell  formats 

49 

move  cells 

50 

name  cells 

DATABASE  FUNCTIONS 


copy  a  database 

create  a  database 

limit  data  type  input 

query  database 

restrict  data  entry  input 

sort  database 
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56 

57 

58 

59 

60 


EDITING 


change/set  font 

draw  lines 

edit  data 

insert  page  breaks 

n 

search  and  replace 

set  protection 

shade 

transpose 

use  undo 

61 

62 

63 

64 

65 

66 

67 

68 
69 
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GRAPHING 


77 

add  text  1 

— 

78 

choose  graph  type 

79 

create  a  graph 

80 

customize  series 

81 

hide  graph 

82 

ifisert  graph  into  spreadsheet 

83 

modify  X  and  Y  axis 

84 

name  graph 

85 

{Mrint  graph 

■ 

86 

specify  series 

87 

use  fast  graph 

88 

use  overall  to  modify  graph  (3d,  colors,  grids) 

89 

use  the  annotator  to  annotate  graphs 

90 

use  the  annotator  to  create  text  graphs 

91 

view  graph 

--  - 
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t  spreadsheet 


110 1  save  and  restore  a  workspace 


111  select  meau  trees 


112 'shortcut  keys 


113 1  sort^ind,  and  extract  database  records 


114 1  use  adjust  printer  command 


115  use  the  built-in  SQZ!  function 


116 ;  use  the  file  manager _ 

117  use  transcript _ 

118 1  use  what -if 


119  xtract  into  separate  file 


OTHER  GENERAL  OPERATIONS 

120  create  and  enter  formulas _ 

121  enter  data _ 


122  enter  snreadsheet 


-*■  -  -  - 

123  exit  spreadsheet _ 

124  preview  spreadsheets  to  screen _ 

125  search  and  replace  spreadsheet  values 

126  use  the  pull  down  menu 
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SETUP  OPTIONS _ 

136 '  modify  colors _ 

137  modify  display  mode _ 

138  modify  graphics  quality _ 

139 1  modify  mouse  pallet 


140  modify  startup  options. _ 

141  set  global  formats _ 

142  set  protection  mode _ 

143  set  recalculation  mode _ 

144  set  uodo  command 


145  set\cbange  hardware  options 

146  update  (save)  setup  changes 


WINDOWS 
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Appendix  B:  The  Evaluation  Form  for  the  Post-Course 
Spreadsheet  Assignment 


Spreadsheet  Project  Evaluation  Form 


Student '  Name : 


Relative  Achievement 


LO  < - >  HI  Weight 

Achieved  project  goals  X  3  = 

Creativity  X  1  = 

Documentation  (on/off  disk)  _  X  1  = 

Effective  use  of 

spreadsheet  capabilities  _  X  2  = 

Layout/Qrganization  X  2  = 


1.2  3  4  5 

OVERALL  RATING: 
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Appendix  C :  The  Post-CH?T  290  Student  Survey 


Poet  CBI  Student  Survey  Ctrl  #  _ 

Thank  you  for  completing  the  Introduction  to  Spreadsheets  Computer 
Based  Instruction  (CBI)  program,  and  for  taking  time  to  fill  out  this 
survey.  The  control  number  you  find  at  the  top  of  this  page  is  NOT 
controlled  by  name,  and  is  for  analysis  purpose  only.  No  individual 
data  will  lie  released.  All  analyses  and  findings  will  be  based  on  group 
trends,  NCTT  individual  data. 

The  purpose  of  this  survey  is  to  collect  your  opinions  on  the 
effectiveness  and  efficiency  of  the  CBI  program,  and  to  solicit  youi‘ 
comments  and  recommendations  for  future  improvements.  Please  place  a 
tick  mai-k  on  the  rating  scales  at  the  point  you  feel  best  answers  each 
particu.L  '  question. 

1.  The  objective  of  the  CBI  program  was  to  teach  you  some  basic- 
spreadsheet  featui'os  and  functions.  How  EFFECTIVE  was  the  CBI  program 
in  meeting  its  objective? 

Completely  I - 1 - * - 1 - 1 - 1 - 1  Highly 

Ineffective  1234567  Effective 

2.  How  many  sessions,  and  approximately  how  much  time  per  session  did 
you  spend  with  the  CBI  program?  NCTE:  Include  ONLY  the  time  you  used 
the  CBI  program.  Do  NOT  include  the  time  you  spent  on  the  projects  or 
things  external  to  the  CBI  progi'am. 

_  sessions  at  approximately  _  MINITES  per  session 


3.  Please  rate  the  CBI  tutorial 

program  in 

the 

fol lowing 

areas ; 

Ease  of  use.- 

Elxtremely 

: — 1 - 1 — 

-1 — 

H — 

— 1 — 

t 

Extremely 

Difficult 

1  2  3 

4 

5 

6 

7 

Easy 

The  author’s 

Unclear  and 

: — 1 - 1 — 

H - 

-H - 

— 1 — 

1 

1 

Clear  and 

writing  style.- 

Hard  to 

1  2  3 

4 

5 

6 

7 

Easy  to 

Understand 

Understand 

Value  in  helping 

you  do  the 

Not  Valuable 

; — 1 - 1 — 

H - 

-1 — 

— 1 — 

— ; 

Extremely 

FINAL  project: 

at  All 

1  2  3 

4 

5 

6 

7 

Valuable 

4.  Philosophically,  how  do  you  feel  about  CBI  as  a  leeiming  tool; 


Ebctremely 

Negative 


i - ' - 1 - 1 - 1 - 1 - :  Extremely 

1  2  3  4  5  6  7  Positive 


5.  Assuming  the  course  material  were  appropriate  for  CBI.  should  other 
courses  at  ^IT  be  tau^t  by  CBI? 

YES  _  NO  _ 

6.  Use  the  space  below,  the  back  of  this  sheet,  or  a  separate  sheet  to 
make  suggestions  or  comments  on  problems,  or  "bugs"  you  encountered  when 
using  the  CBI  program.  Please  be  specific  as  possible. 


7.  Use  the  space  below,  the  back  of  this  sheet,  or  a  separate  sheet  to 
make  suggestions  or  comments  on  the  CBI  course,  or  CBI  in  general .  . 


8.  Which  of  the  following  types  of  computer  users  would  you  recommend 
take  the  CBI  course  (check  all  that  apply); 

_  inexperienced  computer  users 

_  moderately  experienced  computer  users 

_  very  experienced  computer  users 

_  none  of  the  above 
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Appendix  D:  The  Major  Problems  that  the  Students  Reported  Relating  to 
Use  of  the  CBI  Software 

Problem  1:  The  format  for  the  screen  default  on  the  newer  version  of 

(Xiattro  Pro  (version  3.0)  was  not  compatible  wiLn  the  format 
of  the  CBI  program  screens. 

Solution:  The  students  were  instructed  how  to  change  the  screen 

default  on  the  newer  Quattro  Pro  version,  making  it 
compatible  with  the  CBI. 


Problem  2:  Some  students  had  problems  loading  the  CBI  on  the  School  of 
Systems  and  Logistics’  computers  due  in  part  to  the  local 
area  network  setup. 

Solution:  Printed  instructions  were  distributed  providing  students 

step-by-step  instructions  on  how  to  load  the  (2BI  on  the 
local  area  network. 


Problem  3:  Students  could  not  get  the  (Xiattro  Pro  software  that  was 

loaded  on  the  School  of  Systems  and  Logistics  Computers  to 
print  graphs. 

Solution:  The  School  of  Systems  and  Logistics  computer  managers  were 

notified  and  attempted  to  fix  the  problem.  The  problem  was 
not  solved  prior  to  completion  of  the  CBI  course. 

Problem  4:  A  few  students  could  not  successfully  load  and  start  the  CBI 
program. 

Solution:  The  students  >dio  raised  the  problem  were  assisted 

individually  by  the  CBI  author  in  lo.ading  and  starting  the 
CBI  program . 
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Appendix  E:  Post-course  Spreadsheet  Assignment 


QMGT  290  -  Spreadsheet  Project 

1.  Create  a  spreadsheet  containing  your  ed  plan.  (See  LSOI  50-1, 
Attachments  in  your  1992S/D  Graduate  Programs  Handbook  if  your 
advisor  hasn't  given  you  a  copy  of  your  ed  plan.)  Include 
appropriate  column  headings. 

2.  Use  the  spreadsheet  to  compute  and  display  the  following  pieces 
of  data: 

a.  number  of  credit  hours  in  each  term  (quarter) 

b.  total  number  of  credit  hours 

c.  number  of  credit  hours  by  department 

d.  percentage  of  total  credit  hours  by  department 

e.  total  thesis  hours 

f .  percentage  of  total  hours  dedicated  to  thesis 

NOTE:  Do  not  "hard-code"  these  formulas,  i.e.  Perform  the 
calculations  in  such  a  way  that  no  formulas  would  have  to  be 
changed  if  you  replaced  a  course  in  one  department  with  a  course - 
from  another  depai'tment. 

HINT:  You  may  want  to  facilitate  calculation  of  hours  by 
department  by  creating  a  separate  work  area  in  your  spreadsheet 
which  picks  out  the  hours  relevant  to  each  department  using  @IF 
functions. 

3.  Create  graphs  which  show: 

a.  workload  (credit  hours)  by  quarter  (bar  chart) 

b.  workload  (credit  hours)  by  department  (pie  chart) 

4.  Print  the  spreadsheet.  Include  a  header  with  your  name  and 
section.  Beginning  two  lines  below  the  end  of  the  spreadsheet  (but 
beginning  on  the  same  page),  display  the  cell  contents  for  each 
item  computed  in  #2  eibove.  i.e.  Show  me  an  example  of  each  of  your 
formulas. 

5.  Print  both  of  your  graphs  on  a  single  sheet  of  paper. 

6.  Staple  your  sheets  together  and  turn  in  NIT  Monday,  24  Jun  91. 
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Appendix  F :  Examples  of  CBI  Lesson  Frames 


The  SPREADShEETT  WINDOW  is  the  rectangular  area  on  which  this  text 
is  displayed.  It  is  made  up  of  cells  where  you  enter  display  and 
organize  information.  Each  cell  has  ein  address- which  includes  a 
vertical  column  letter  and  a  horizontal  row  number.  The  column 
letter  is  found  just  above  the  spreadsheet  window.  The  row  number 
is  just  left  of  the  spreadsheet  window.  This  spreadsheet  window 
shows  columns  I-P  and  rows  39-58.  The  entire  spreadsheet,  however, 
has  256  columns  (lettered  A-Z  then  AA-A2  BA-BZ  etc.  up  to  IV)  and 
8192  rows. 

TAB  to  continue  SHIFT  TAB  to  backup  ALT  T  to  test  ALT  Q  to  quit 


TRY  THIS  ->  Move  the  cell  selector  to  BA133.  Notice  the  input 
line  displays  the  formula  (+AY133+Ayi35)  while  the  cell  displays 
the  formula  results  (160).  The  formula  adds  the  contents  of  AY133 
(100)  to  the  contents  of  AY135  (60).  While  the  cell  selector  is 
still  at  BA133  press  the  F2  key  to  enter  the  edit  mode.  Experiment 
with  the  formula  contents  using  the  four  arithmetic  operators.  If 
you  get  an  error  message  (ERR)  enter  a  corrected  formula. 


TAB  to  continue  SHIFT  TAB  to  backup  ALT  T  to  test  ALT  Q  to  quit 


Answer  each  question  by  pressing  the  letter  corresponding  to  the 


correct  answer. 


Question  1.  The  spreadsheet  window  is  made  up  of  _  where  you 

may  enter  display  and  organize  infonnation. 


a.  pallets 

b.  memory  allocation  units 

c.  labels 

d.  cells 


Select  a.  b.  c,  or  d 


The  correct  answer  is  D. 

The  spreadsheet  window  is  made  up  of  cnJS  where  you 
may  enter  display  and  organize  infonnation. 


Press  TAB  to  continue 
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Appendix  H:  A  Survey  of  the  Students  vrt^o  had  Completed 
□<GT  290  Prior  to  the  Research  Period 


Survey  Questionnaire  Instructions 

1.  Based  on  your  successful  completion  of  the  summer  short  term  course 
Introduction  to  AFIT  Computer  Systems  (C^<GT  290) .  you  are  requested  to 
participate  in  a  research  project.  Your  responses  to  the  items 
contained  in  this  survey  questionnaire  will  used  to  assess  the 
availability  of  the  computer  resources  required  for  a  computer-based 
instruction  course. 

2.  Anonymity  is  assured  as  no  identifying  data  is  requested. 

Individual  responses  will  not  be  released. 

3.  If  you  feel  the  survey  does  not  adequately  address  any  if ^ues. 
please  feel  free  to  comment  or  add  additional  information. 

4.  Please  place  your  completed  questionnaire  in  Russ  GREENE'S  mail 
(distribution)  box  today,  or  as  soon  as  possible. 


QMGT  290  Former  Student  Survey 


1 .  Co  you  own  a  perisc^nil  ct^mputer'? 

YES  _  N(j  _  (If  NO,  skip  to  question  6) 

2.  Please  specify  the  typ*^  of  computer (s)  you  own. 

DOS  _  MACINTOSH  OTHER  _ 

3.  Did  you  purchase  a  computer  after  arrival  at  AFIT? 

YES  _  NO  _ 

4.  Was  your  computer  set  up  otnd  ready  for  use  by  the  beginning  of  the 
summer  short  term? 

YES  _  NO  _ 

5 .  Did  you  use  your  computer  for  the  summer  shor i  term  course  290 . 

'Introduction  to  AFIT  Computer  Systems"? 

'fES  _  NO  _ 
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6.  Do  you  feel  AFIT's  Z-248  personal  computers  are  adequately  available 
for  your  use? 

YES  _  NO  _  (If  NO,  please  exolain) 


(continue  on  reverse  if  necessary) 

7.  Did  you  use  an  automated  spreadsheet  program  before  coming  to  AFIT? 
YES  _  NO  _  (If  yes,  please  specify  program  name) 


8.  Did  you  own  an  automated  spreadsheet  program  before  coming  to  AFIT? 
YES  _  NO  _ 


9.  Did  you  use  CXiattro  Pro  for  the  (^4(3T  290  (Introduction  to  AFIT 
Computer  Systems)  project (s)? 

•  YES  _  NO' 

10.  If  you  had  the  option  to  take  the  spreadsheet  portion  of  (3MGT  290  by 
the  ctiiTent  classroom  method  or  by  a  tutorial  program  on  a  computer 
disk  (computer  based  instruction),  which  method  would  you  prefer-' 

Classroom  _  Tutorial  Disk  _  No  Preference  _ 

11.  Have  you  ever  taken  a  course  by  computer  based  instruction? 

YES  _  NO  _ 

12.  If  you  have  any  comments  or  suggestions  that  would  be  helpful  in 
designing  and  implementing  a  computer  based  instniction  course  for 
the  spreadsheet  portion  of  QMGT  290.  please  write  them  below. 


(continue  on  reverse  if  necessary) 
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Appendix  I :  General  Findings  from  the  Survey  of  Students  who  had 
Completed  CMGT  290  Prior  to  the  Research  Period 


Thank  you  for  your  participation  in  this  survey.  Your  collective 
responses  are  provided  below.  A  special  thanks  for  all  the  constructive 
comments.  Your  collective  (not  individual)  comments  will  be  provided  to  the 
OMGT  290  course  director  for  potential  improvements.  Of  the  135  surveys 
distributed.  89  were  completed  and  returned  for  a  65.9  percent  response  rate. 
Some  response  percentages  do  not  add  to  100  percent  due  to  “no  responses  '  or 
rounding  error.  Once  again,  thanks  for  your  support  and  concern. 


Student  Survey  Collective  Responses 

1.  Do  you  own  a  personal  computer? 

YES  97.8%  NO  2.2%  (If  NO.  skip  to  question  6) 

2.  Please  specify  the  type  of  computer (s)  you  own. 

DOS  93.4%  MACINTOSH  5.5%  OIHER  1.1% 

3.  Did  you  purchase  a  computer  after  arrival  at  AFIT? 

't’ES  49.5%  NO  50.5% 

4.  Was  your  computer  set  up  and  ready  for  use  by  the  beginning  of  the  summer 
short  term? 

YES  64.8%  NO  35^^ 

5 .  Did  you  use  your  computer  for  the  summer  short  term  course  OIGT  290 . 

Introduction  to  AFIT  Computer  Systems”? 

YES  74.7%  NO  24.2% 

6.  Do  you  feel  AFIT's  Z-240  personal  computers  are  adequately  available  for 
your  use? 

'YES  68.8%  NO  28.0%  (If  NO.  please  explain) 

CCMMEKT  TRENDS  FROM  HIGH  TO  LOW  RESPONSE  RATE: 

1.  Computers  are  too  slow. 

2.  Printer  problems  (not  adequate,  ribbons,  working 
condition,  etc.) 

3.  Computers  are  adequate  only  because  most  students  use 
their  own  computers. 

4.  Software  problems  (inadequate,  not  standardized  between 
computers,  slow) 

5.  Computers  are  in  poor  working  condition. 


76 


7.  E)id  you  use  an  automated  sprea<dsheet  progr'ain  before  coming  to  AFIT? 

YES  48.4%  NO  51.6^ 

8.  Did  you  own  an  automated  spreadsheet  program  before  coming  to  AFIT? 

YIB  29.0%  NO  71,0% 

9.  Did  you  use  Quattro  Pro  for  the  QMGT  290  (Introduction  to  AFIT  Computer 
Systems )  pro j ect ( s ) ? 

iES  88.2%  NO  11.8% 

10.  If  you  had  the  option  to  take  the  spreadsheet  pf:)rtion  of  QM(3T  290  by  the 
cuii'ent  classroom  method  or  by  a  tutorial  program  on  a  computer  disk 
(computer  based  instruction),  which  method  would  you  prefer? 

Classroom  51.6^  Tutorial  Disk  31.2%  No  Preference  17,2% 

11.  Have  you  ever  taken  a  course  by  computer  based  instruction? 

YES  36.6%  NO  63.4% 

12.  If  you  have  any  comments  or  suggestions  that  would  be  helpful  in  desigrang 

and  implementing  a  computer  based  instruction  course  for  the  sprea<asheet 
portion  of  290.  please  write  them  below. 

COMMENT  TRENES  FROM  HIGH  TO  LOW  RESPONSE  RATE: 

1.  Make  CBI  user  friendly,  especially  for  computer  ine:<periencefl  students. 

2.  Use  C3I  to  supplement,  not  replace  instructor. 

3.  Keep  in  mind  the  vast  differences  in  student  computer  experience  levels. 

4.  Teach  more  DOS  arni  computer  basics.  Do  not  assume  students  kni:>w  CCS. 

5.  Allow  students  to  test  out  of  QMGT  290. 

6.  Teach  more  about  spreadsheets  to  better  prepare  students  for  follow-on 
courses . 

7.  Concentrate  spreadsheet  instruction  on  practical  rather  than  abstract 
applications. 

8.  Make  CBI  self-paced. 

9.  Build  to  one  major  graded  project  with  smaller  sub-projects. 

10.  Use  Lotus  123  computer-assisted  instiuction  modified  for  Oiattro  Pro. 

11.  Eliminate  QMGT  290. 

12.  Notify  incoming  students  prior  to  arrival  of  the  software  used  at  AFIT. 

13.  Use  an  accompanying  CBI  workbook  with  the  CBI  lessons. 
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Appendix  J :  Survey  of  the  Incoming  Students  who  would  Take  (MjT  290 
During  the  Research  Period  (Pre-QMGT  290  Survey) 


WELCOME  to  AFIT 

One  of  your  first  courses  at  AFIT  will  be  C^4GT  290,  Introduction 
to  AFIT  Computer  Systems.  Offered  in  the  summer  review  term,  this 
course  will  help  you  "get  up  to  speed"  on  AFIT's  computer  resources  used 
in  other  courses.  This  summer,  you  and  other  members  of  Class  92  will 
help  us  test  a  new  method  of  course  delivery.  Specifically,  some  of 
your  class  will  study  the  spreadsheet  portion  of  QMGT  290  in  the 
traditional  classroom  method,  while  others  will  study  spreadsheets 
through  a  new,  self-paced,  tutorial  on  computer  disk.  At  the  erd  of  the 
term,  we  will  compare  the  effectiveness  of  the  two  methods  and  user's 
preferences.  If,  as  we  believe,  the  convenience  of  a  self-paced 
tutorial  enhances  learning  and  helps  users  better  manage  their  time. 

AFIT  may  adapt  other  courses  for  computer-based  instruction. 

Let  us  empnasize  that  this  project  is  designed  to  ensure  effective 
learning  for  students  in  both  groups.  Both  versions  of  the  course  will 
cover  the  same  material,  and  instructor  support  will  be  equally 
available  to  everyone.  Since  QMGT  290  is  graded  only 
’ 'Sat isf act ory /Unsat isfactory . "  no  one's  final  grade  will  be  negatively 
affected  by  either  method  of  instnjction.  Students  in  the  tutorial 
group  may  use  their  home  computers  or  use  AFIT's  computers  during  non¬ 
class  hours. 

To  create  a  valid  research  design,  we  need  to  ensure  that  the  two 
gi'oups  are  "mirror'  images"  of  each  other  in  terms  of  prior  knowledge  of 
computers,  spreadsheets,  and  related  variables.  The  attached 
'ijuestionnaire  will  help  us  achieve  that  comparability.  Here  is  the 
procedure : 

-  "fou  complete  the  questionnaire,  including  a  statement  of  your 
preference  for  which  group  you  would  like  to  be  in. 

-  You  return  the  questionnaire  to  the  box  on  the  counter  of  LSG 
(Room  316)  BEFORE  YOU  LEAVE  THE  BUILDING  TODAY. 

-  The  researcher  (iIPT  Greene)  analyzes  the  questionnaires  and 
establishes  the  two  groups  before  the  experiment  starts.  Only 
CPT  Greene  will  sec  your  responses,  and  all  responses  will  be 
treate<i  as  confidential.  Since  each  <7i'oup  will  contain 
experienceti  and  novice  computer  users  and  experienced  and  novice 
spreadsheet  users,  the  faculty  will  not  know  how  you  answered 
any  questions  or  the  basis  of  assignment  into  either  gi'oup. 


Although  you  must  ta)<e  QMGT  290.  youi'  pai'ticipation  in  the 
research  is  voluntary.  It  you  want  to  be  a  candidate  for  the  tutorial 
method,  simply  indicate  that  fact  on  the  questionnaire  and  you  will  be 
excused  from  class  on  the  "CXiattro  Pro"  class  days.  However .  to  ensure 
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the  validity  of  the  research  (and  to  help  the  classrtxim  insLri.ictors  meet 
everyone's  needs),  it  is  very  important  that  we  receive  a  completed 

questionnaire  from  you.  Be  sure  to  submit  your  completed  quest ionnait'e 
to  LSG  before  you  leave  today. 

Thank  you  for  helping  AFIT  help  you. 


Course  Director,  C^<GT  290 


PART  I:  Demcxarat^ics 


1 .  Last  Name  _  F irst  Name  _  MI  _ 

2.  Rank  or  Grade;  _ 

3.  Please  indicate  your  branch  of  service: 

a.  Air  Force  b.  Navy  c.  Government  Civilian 

d.  Army  e.  Marine  f.  International  Student 

g.  Other  (please  specify)  _ 

4.  Please  fill  in  the  education  level (s)  you  have  completed.  Specify 
the  subject  area,  degree (s)  ear'ned,  and  approximate 

comp letion/graduat ion  date  (month  and  year)  in  the  space  provided 
(continue  on  the  back  of  this  sheet  if  necessary) ; 

a .  undergraduate  degree ( s )  subject  area ( s )  _ 

degree (s)  earned  _ 

year  _ 

b.  igiaduate  degree (s)  subject  area(s)  _ 

degree (s)  earned  _ 

year  _ 


c.  others  (for  example  technical  schools,  correspondence  courses, 
significant  mi  1 itary /government  courses,  etc.) 


(Continue  on  reverse  if  necessary.) 
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5.  Please  indicate  your  completed  number  of  years  of 
govemment/mi  1  i tary  service : 

_  years 

6.  Which  AFIT  graduate  program  are  you  enrolled  in  (for  example.  GEM. 
GLM.  GCM.  GIR.  GSM.  GCA,  etc.)? 


PAfTT  II:  Ganputer  Ebcperience 

Instructions  for  Part  II;  For  the  purpose  of  this  quest iorma ire,  a 
"personal"  computer  (PC)  is  a  stand-alone  microcomputer  consisting  of  a 
central  processing  unit  cIPU) .  a  visual  display,  a  keyboard  input 
device,  and  one  or  more  disk  drives.  Personal  computers  can  also  be 
referred  to  as  "desktop."  "home."  "laptop."  "notebwk."  or  "portabie" 
computei's.  Some  commercial  examples  of  PC's  are  IBM  (sometimes  referred 
to  as  MS-DOS  systems).  Apple.  Commodore,  and  compatibles.  In  this 
questionnaire,  references  to  "computers"  that  are  not  preceded  by 
"personal”  refer  to  al 1  computers-,  including  personal  computers. 

EScamples  include  mainframes,  minicomputers,  eind  personal  ccanputers. 

Some  of  the  questions  will  ask  you  to  rate  an  item  on  the  following 
sea  1 e : 

Completely  Highly 

Inexperienced  i - 1— — i - 1 - ¥ - 1 - :  Experienccxi 

1  2  3  4  5  6  7 

Place  a  tick  mark  on  this  scale  at  the  point  you  feel  best  describes 
your  experience  level  for  the  pai'ticular  question  topic.  Below  is  an 
example  of  how  to  correctly  mark  the  scale  for  a  person  whe  feels  aimt^st 
completely  inexperienced  with  the  particular  topic. 

Completely  Highly 

Inexperienced  I - • - • - 1 - 1 - 1 - ;  Ebcperienced 

1  2  3  4  5  6  7 

Hardware 

L .  Do  you  own  one  or  more  persona  1  computer's? 

YES  _  (If  yes.  how  many?  _ ) 

NO  _  (If  NO.  skip  to  question  5.) 

2.  Is  (are)  your'  persona  1  computer! s)  .- 

IBM  or  Compatible?  _ 

APPLE  or  Compatible?  _ 

Other  (please  specify)  _ 
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3.  In  what  year  did  you  acquire  your  personal  computer(s) ? 


Year ( s )  _ 

4.  Will  you  use  your  own  personal  ccmputer  for  the  summer  short  term? 
YES _  NO _ (If  NO,  please  specify  why.) 


(Continue  on  reverse  if  necessary.) 

5.  Did  you  use  a  computer  prior  to  enrolling  at  AFIT? 

■fES  _  NO  _  (IF  NO.  skip  to  PART  III.) 

6.  In  what  year  did  you  first  use  a  computer? 

Year  _ 

7.  Which  of  the  following  best  describes  your  typical  frequency  of  use 
of  computers? 

Very  little  _ 

Little  _ 

A  moderate  amount  _ 

Much  _ 

Very  much  _ 

8.  Place  a  check  mark  beside  each  of  the  following  items  that  you  could 
accui'ately  define; 

ANSI 
CD  ROM 
CPU 

floppy  disk  drive 
input /out put  card 

9.  Consider  for  a  moment  your  overall  experience  with  computeis.  How 
would  you  rate  your  computer  experience? 

Completely  '  Highly 

Inexperienced  ; - ' - 1 - 1 - ' - 1 - :  Experienced 

1  2  3  4  5  6  7 

10.  Please  describe  the  education  or  training  you  have  had  in  computer 
use.  If  none,  enter  "none." 


LPT  1 
modem 

motherboard 

mouse 

serial  port 


Software 


11.  Please  describe  any  computer  software  that  you  have  personally 
written  including  the  language  and/or  program.  If  none,  state  "none.’ 


(Continue  on  reverse  if  necessary.) 

12.  Please  rate  your  experience  with  the  following  software  categories: 

Con^letely  Highly 

Inexperienced  Ebcperienced 

CAD  programs . : - 1 - 1 - 1 - 1 - 1 - i 

database  programs . : - 1 - 1 - 1 - 1 - 1 - 1 

draw,  paint,  or  graphics  programs.  .  ; - 1 - 1 - 1 - 1 - 1 - ; 

entertainment  programs . : - 1 - 1 - 1 - 1 - 1 - 

math  or  statistical  programs  ....  : - 1 - 1 - 1 - ^ - 1 - ; 

spreadsheet  programs . 1 - 1 - 1 - 1 - 1 - • - 1 

telecommunications  programs . 1 - 1 - -i - 1 

ut  1 1 1  ty  programs . i - 1 - 1 - 1 - ^ 1 

woixl  process  irg  programs . f - 1 1 - : 

1  2  3  4  5  6  7 

Others  you  feel  experienced  with  (please  specify) 

;• - 1 - +  — I - 1 - 1 - : 


: - 1 - 1 - 1 - 1 - H — ; 

1  2  3  4  5  6  7 


PART  III;  Spreadsheet  Experience 

1 .  Have  you  used  an  automated  spreadsheet  program? 

YES  _  NO  _  (If  NO.  skip  to  PART  IV) 

2.  What  is  the  name  of  the  spreadsheet  program (s)  you  have  used? 


3.  Please  describe  the  nature  of  use.  or  for  what  purpose  you  used 
spreadsheet  programs. 


(Continue  on  reverse  if  necessary.) 
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4.  Do  you  own  an  automated  spreadsheet  progi'am? 

YES  _  NO  _  (If  NO.  skip  to  question  7) 

If  YES.  please  specify  the  program  name:  _ 

5.  When  did  you  obtain  your  automated  spreadsheet  program? 

Year _ 

6.  For  what  purpose  did  you  obtain  your  automated  spreadsheet  program? 


(Continue  on  reverse  if  necessai-y.) 

7.  When  did  you  first  use  an  automated  spreadsheet? 

Year  _ 

8.  Which  of  the  following  best  describes  your  typical  frequency  of  'use 
of  spreadsheets? 

Very  little  _ 

Little  _  • 

A  moderate  amount  _  • 

Much  _ 

Very  much  _ 


9.  Please  indicate  your  experience  level  with  the  following  spreadsheet 


functions  or  concepts: 

(Completely  Hic^ily 

Inexperienced  Eicperienced 

-  blocking . I - • - • - 1 - h — i - ! 

-  database  functions  .  ; - • - ' - ' - • - ' - 

-  editing . I - 1 - • - • - 1 - • - ■ 

-  file  management.  .  .  i - ' - 1 - • - 1 - 1 - : 

-  graphing . : - 1 - 1 - 1 - 1 - -< - 1 

-  math  functions  .  .  .  ! - 1 - 1 - 1 - 1 - 1 - 1 

-  printing  spread¬ 

sheets  . ; - 1 - 1 - 1 - 1- - 1 - i 

-  row  and  column 

operations . : - ' - 1 - 1 - • - f - 1 

-  using  formulas  .  .  .  ! - 1 - 1 - • - * - 1 - 

-  using  macros  ....  I - 1 - 1 - 1 - 1 - 1 - 1 

-  windows . I - 1 - 1 - 1 - 1 - 1 - ; 

-  writing  formulas  .  .  ! - 1-  — • - 1 - ' - * - 

-  wr  iting  macros  .  .  .; - ^ - 1 - h - h - 1 - : 


1  2  3  4  5  6  7 
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10.  Overall,  how  would  you  rate  your  autcanated  spreadsheet  experience? 


Completely  Highly 

Inexperienced  1 - 1 - 1 - 1 - 1 - 1 - 1  Experienced 

1  2  3  4  5  6  7 

11.  Have  you  ever  taken  a  class  in  automated  spreadsheet  use? 
YES  _  NO  _ 

If  YFS.  please  specify  course,  length,  and  date;  _ 


(Continue  on  reverse  if  necessary.) 


PART  IV:  Spreadsheet  Knowledge 

Instructions  for  PART  IV;  The  results  of  this  test  will  be  used  only 
for  this  research  project.  Individual  responses  will  not  be  released. 
Please  circle  the  answer  that  best  fits  the  blank.  It  is  important  t.hat 
you  DO  NOT  GO'ESS.  If  you  are  not  reasonably  sure  of  an  answer,  mark  the 
"1  am  not  sure"  block. 

1.  A  spreadsheet  cell  has  an  address  or  location  which  is  made  up  of 

_  and  _  coordinates. 

A.  block,  column 

B.  block,  line 

C.  row.  column 

D.  row.  block 

E.  I  am  not  suire 

2.  A  spreadsheet  block  nomally  consists  of  _ . 

A .  more  than  one  cell. 

B.  two  or  more  embedded  macros. 

C.  a  series  of  graphical  interface  commands  linking  the  spreadsheet 
to  a  graphical  block  display  area. 

D.  a  set  of  special  blocking  codes. 

E.  I  am  not  sure 

3.  In  an  automated  spreadsheet,  pressing  the  right  arrow  key  while  the 

cursor  is  in  a  cell  that  contains  a  formula  will  _ . 

A.  execute  the  next  sequenced  autorecalcmlation. 

B.  move  the  cursor  one  cell  to  the  right. 

C.  do  nothing  unless  used  in  combination  with  another  key. 

D.  pr'ompt  the  user  for  a  cell  value  or  label. 

E.  1  am  not  sur'e 
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4.  Tlie  pur[x'>3e  of  a  Lujor-deiinoj  macro  13  lo  _ . 

A.  save  all  related  3preack:i leets  under  the  same  macro-group  name. 

B.  create  hidden  definition  blocks  which  are  only  visible  when 
editing. 

C.  allow  another  user  to  simultaneously  logon  to  the  same 
spreadsheet  program  (assuming  both  spr'encisheet  macros  modes  .are 
compatible) . 

D.  allow  the  user  to  consolidate  many  keystrokes  into  a  smaller 
number  of  keystrokes. 

E.  I  am  not  sure 


5.  in  most  automated  spreadsheet  programs,  _  would  compute  the 

square  of  54. 

A.  54'2 

B.  54  */2 

C .  54  (asqr  ( 2 ) 

D.  54'l'reqr(2) 

E.  I  am  not-  sure 


6.  In  an  automated  spreadsheet,  autorecalculation  refers  to  the 

spreadsheet's  ability  to  automatically  _ . 

A.  move  all  relevant  mathematical  formulas  to  a  user  defined 
location. 

B.  update  formulas  and  calculations  of  other  spreadsheets  saved 
under  tlie  same  igroup  name. 

C.  update  formula  or  calculation  results  when  referenc.=^  cell  values 
are  chang»=^i . 

D.  allow  the  user  to  redefine  referenced  formula  and  calculation 
values  prior  to  loading  the  spreadsheet. 

E.  I  am  not  sui  e 


7.  To  enter  data  into  a  empty  (unprotected  or  unlocked)  cell,  you  move 
the  cursor  to  the  cell,  _ . 


A.  and  enter  the  desired  data. 

B.  define  the  data  worksheet  area,  and  then  enter  the  data. 

C.  change  the  flag  indicator  on  the  cell,  and  then  enter  the  data. 
E).  change  the  definition  of  the  cell  to  a  block,  and  then  enter  the 

data . 

E.  I  am  not  sui'e 
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8.  In  most  spreadsheet  programs,  entering  a  data  string  20  chcur'acters 

long  into  a  cell  of  width  8.  results  in  the  spreadsheet _ . 

A.  automatically  changing  the  cell  to  a  block  in  order  to  accept  the 
excess  characters. 

B.  not  acknowledging  the  excess  data  unless  the  user  manually 
changes  the  cell  to  a  block. 

C.  accepting  and  acknowledging  the  excess  characters  even  though 
they  may  not  be  visible. 

D.  automatical ly  storing  the  excess  characters  in  the  adjacent  cell. 

E.  I  .am  not  sure 


PART  V:  C3I  Ebcperience 

Computer  based  instruction  (CBI)  is  an  educational  tool  that  uses 
computers,  computer  programming,  and  human-computer  interfacing  to  aid 
or  conduct  the  education  or  training  of  people.  For  the  purposes  of 
this  questionnaire.  CBI  software  is  any  training  or  educational  computer 
program  that  interactively  engages  in  dialogue  with  the  user  for  the 
purpose  of  training  or  educating.  CBI  can  also  be  referred  to  as 
computer  assisted  instruction  (CAD,  computer  assisted  learning  (CAL), 
computer  based  education,  and  many  others.  Some  examples  of  CBI  include 
interactive  tutorial  programs,  simulations,  or  automated  drill  and 
practice  (multiple  choice  question  and  answer)  programs.  Computerised 
entertainment  programs  (such  as  Nintendo  or  SEGA)  are  not  considered  CBI 
unless  the  express  purpose  of  the  program  is  to  educate  or  train  people, 

1.  Based  on  the  above  definition,  have  you  used  CBI? 

YES  _  NO  _  (IF  NO.  skip  to  PART  VI) 

2.  When  did  you  first  use  CBI? 

Year  _ 

3.  Which  of  the  following  best  describes  your  typical  frecaiency  of  use 
of  CBI? 

Very  little  _ 

Little  _ 

A  moderate  amount  _ 

Much  _ 

Very  much  _ 

4.  Please  describe  the  type  of  CBI  you  used?  (Examples  include 
software  tutorials,  simulations,  automated  question  and  answer 
programs,  automated  testing,  etc.) 


(Continue  on  reverse  if  necessary.) 
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5.  Think  a  moment  about  your  experience  with  CBI.  How  would  you  rate 
your  CBI  experience? 

Completely  Highly 

Inexperienced  1 - 1 - 1 - 1 - 1 - 1 - !  Experienced 

6.  Have  you  ever  written  a  computer  based  instruction  program? 

YES  _  (li  YES.  please  describe  below)  NO  _ 


(Continue  on  reverse  if  necessary.) 


PART  VI:  Completion  Instructions 

Thank  you  for  taking  the  time  to  complete  this  questionnaire.  Youi' 
input  is  important  to  TVFIT  and  this  research  project.  Your  voluntary 
cooperation  is  invaluable  to  the  success  of  this  research.  Please  check 
the  statement  that  applies  to  you: 

_  I  would  prefer  to  participate  in  this  project  as  part  of  the 

computer  based  tutorial  group. 

_ _  I  would  prefer  to  participate  in  this  project  as  part  of  the 

in-class  lecture  group. 
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Appendix  K;  General  aiaracteristics  of  the  Student  Gtoups 


General  Character iat ice  of  the  Student  Groups  Baaed  on  the 
Survey  of  the  Incoming  QMGT  290  Students 


All  Students  CBI  Students  In-class  Students 
%  All  %  CBI  %  In 


Had  an  education  higher 


than  a  BS; 

18 

18% 

10 

22% 

8 

14% 

Average  years  of 

government  service; 

10 

10 

10 

Owns  a  computer: 

77 

75% 

35 

76% 

42 

74% 

Type ;  IBM ; 

66 

64% 

31 

67% 

35 

<Dther : 

8 

8% 

1 

2% 

7 

12% 

Used  a  computer  before 

arriving  at  AFIT; 

100 

97% 

45 

98% 

55 

97% 

Had  prior  computer 

training: 

70 

6a% 

34 

74% 

36 

63% 

Wrote  computer  software: 

33 

32% 

24 

52% 

9 

16% 

Used  a  spreadsheet 

program ; 

74 

72% 

35 

76% 

39 

68% 

Owned  a  spreadsheet 

program: 

45 

44% 

22 

48% 

23 

43% 

Had  prior  spreadsheet 

training: 

20 

19% 

5 

11% 

15 

26% 

Had  ever  used  CBI : 

68 

66% 

33 

72% 

35 

61% 

Had  personally  written 

CBI: 

4 

4% 

2 

4% 

2 

4% 
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rent  and  future  applications  of  AFIT  thesis  research.  Please  return 
completed  questionnaires  to:  AFIT/LSC*  Uright-Patterson  AFB  OH 
45433-6583. 

1.  Did  this  research  contribute  to  a  current  research  project? 
a.  Yes  b.  No 


2.  Do  you  believe  this  research  topic  is  significant  enough  that  It  would 
have  been  researched  (or  contracted)  by  your  organization  or  another 
agency  If  AFIT  had  not  researched  It? 

a.  Yes  b.  No 


3.  The  benefits  of  AFIT  research  can  often  be  expressed  by  the  equivalent 
value  that  your  agency  received  by  virtue  of  AFIT  performing  the  research. 
Please  estimate  what  this  research  would  have  cost  In  terms  of  manpower 
and/or  dollars  If  It  had  been  accomplished  under  contract  or  If  It  had 
been  done  In-house. 

Man  Years _ _  $ _ _ 


4.  Often  It  Is  not  possible  to  attach  equivalent  dollar  values  to 
research,  although  the  results  of  the  research  may.  In  fact,  be  Important. 
Whether  or  not  you  were  able  to  establish  an  equivalent  value  for  this 
research  (3  above),  what  Is  your  estimate  of  Its  significance? 

a.  Highly  b.  Significant  c.  Slightly  d.  Of  No 

Significant  Significant  Significance 
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